Networking

Two interfaces, deliberately. The 10 gigabit wired port carries everything. The wifi interface exists as a fallback, and it earned its place.

enP7s7   192.0.2.30/24   Realtek RTL8127, 10 Gb/s full duplex
wlP9s9   198.51.100.30/24   MediaTek, DHCP, separate VLAN

Both are NetworkManager-managed through netplan. The wired connection carries an autoconnect priority of -999, which reads oddly until you realise the intent: wifi should never win the default route while the wired link is up, and the negative priority makes that ordering explicit rather than accidental.

The 10 gigabit interface has a history

The wired port failed to come up after reboots during the machine’s first weeks. The wifi fallback was configured precisely because of this, and it is the reason the box stayed reachable through it.

The driver is r8127, shipped inside the NVIDIA kernel package rather than built through DKMS. Alongside it, DGX OS installs a package whose entire payload is a single modprobe line:

$ cat /etc/modprobe.d/nvidia-spark-r8169.conf
alias pci:v000010ECd00008127sv*sd*bc*sc*i* r8127

A vendor does not ship a package containing one alias unless that alias is solving a known binding problem. The stock r8169 driver will claim this device and handle it badly; the alias forces the correct r8127 driver.

The timeline is suggestive. Failures occurred on kernel 6.17.0-1026. The machine upgraded to 6.17.0-1029 and has held a clean 10 Gb/s link since, with zero carrier-loss events across the retained journal.

Not proven fixed

That is one reboot on the newer kernel, not a controlled test. Journal rotation removed the logs from the failing period, so the original failure signature was never captured. The honest position is that the symptom has not recurred, on a machine that has not been rebooted much.

Because the driver lives inside the kernel package, any kernel upgrade can change this behaviour in either direction. After any kernel update, check:

ip -br link show enP7s7
sudo ethtool enP7s7 | grep -E "Speed|Link detected"
journalctl -b -u NetworkManager | grep enP7s7

The journal is persistent here, so evidence from a future failure will survive the reboot that caused it.

Model downloads are rate limited on purpose

Traffic from this VLAN to Cloudflare-backed model storage is policed to roughly 400 Mbit/s per address.

This is not a conservative default, it is a scar. The first round of model pulls saturated the uplink hard enough to take out streaming video and make the monitoring stack unreachable from elsewhere in the house. A 400 GB model pulled at full 10 gigabit line rate is a denial of service against your own network, delivered by a machine that is technically doing exactly what you asked.

The practical consequence is that large pulls take a while and that is fine. Do not try to defeat the policer. If a pull needs to be faster, the right conversation is about scheduling it overnight, not about removing the limit that keeps everything else usable.

Monitoring

A Checkmk agent listens on 6556 and Telegraf runs alongside it, both feeding the existing monitoring infrastructure. NVIDIA’s own DGX Dashboard service runs on port 11000, bound to localhost, so reaching it means an SSH tunnel:

ssh -L 11000:localhost:11000 your-spark

GPU metrics come from nvidia-smi rather than from any Blackwell-native telemetry, which is worth knowing when a metric looks wrong or missing.