Selfhosted @lemmy.world Dax87 @forum.stellarcastle.net 1y ago

How would you resolve this issue: Your server's internet randomly becomes inaccessible until reboot. (Ubuntu 22.04, Netplan)

I've been having this issue very sporadically (sometimes a couple times a week, sometimes once a month). I'm curious as to how the more veteran folk here would try and narrow down the cause of this issue.

I can provide more info if needed!

Edit: More Info:

Using a static IP (no DHCP) through Netplan.

35 comments

Check if there's some weirdness in IP allocation. A reboot can cause DHCP to give it a new one that works as long as it does, then fail on some weird collision.
- I forgot to mention that I'm not using DHCP, statically allocating a local IP
  
  Is there a DHCP server at play? Is the static IP outside of the DHCP range? This does sound like a typical IP collision.
  
  There might still be another device on the network that is using DHCP and is getting an IP that conflicts. Do you have any visibility on the rest of the network, IP addressing, DHCP leases etc? I would check there first for a potential easy fix.
Honestly I would replace Ubuntu with an actual operating system designed for servers.
- tbf i'm using ubuntu server because I've been using it on and off for a decade. Overall I've had no major issues. And its long-term support is one of the longest.
  
  There's the thought of going RHEL-based which I have experience with from work too, so that is an option in the long run.
check that nameservers are specified in /etc/resolv.conf
- Good idea. I'll check that
  
  if ping fails with the name but works with 8.8.8.8 then you're losing DNS. set the nameservers locally on the server if you're using a static IP.
Are you able to access the server's Linux shell via KVM or out-of-band management during the Internet outage? If so, I would first check the kernel log for any errors. I'd then follow up with a PING to the local gateway within the same IP subnet to validate Ethernet connectivity. After that I'd start performing PINGs and traceroutes to subnets beyond the local gateway to see what's going on with tcpdump capturing all interface traffic in the background.
- I am able to access the Linux shell locally or on the local area network. From within I can ping the gateway, but nothing beyond that.
  
  Is there a tracert equivalent for Ubuntu?
  
  I will say I get this error when trying to ping www.google.com:
  
  ping: unknown host
  
  The tracert equivalent that I use would be traceroute or mtr. If you can get a response from the gateway and access the server locally, it would indicate an IP routing issue as local subnet traffic is working.
  
  Instead of pinging a domain name, what happens if you ping a public IP like Google DNS (8.8.8.8)? The error you're seeing could be related to a DNS resolution issue although that should not affect access to your server.
I have a TP-Link HS300 that connects to a AP running openwrt, at any moment I can wireguard to AP and use Kasa APP to turn on/off any machine (not the AP though).

There is a Python package for kasa so one can use openwrt machine to detect if machine is down and automatically switch on and off if needed.
- I was thinking of something like a local script to reboot the PC once internet isn't reachable after 5 min or so, but only as a stopgap/workaround until I figure out the issue.
Can you pinpoint the exact moment when the internet cuts out? Have you checked your logs (dmesg, /var/log/syslog, journalctl, etc) around that time for anything weird?
- Unfortunately not. It's really exposing my need for a health monitoring service like TIG stack or something equivalent.

35 comments