Networks

References

• systemd-networkd - from archlinux, with reference to systemd-nspawn for container networking. -- interesting note: "it is possible to run Docker containers inside an unprivileged systemd-nspawn container with cgroups v2 enabled". But from an old talk:
  • Creating containers with systemd-nspawn - It is targeted at "building, testing, debugging, and profiling", not at deployment. systemd-nspawn uses the same kernel APIs that the other two tools use, but is not a competitor to them because it is not targeted at running in a production environment.

Gone are the good 'ole days of using /etc/network/interfaces to manage basic networking stack configurations. There seems to be an explosion of alternate ways, each stepping on each other's toes: /etc/network/interfaces, NetworkManager, NetPlan, systemd-networking, ...

A problem I had the other day was where a new installation of an espressobin had NetworkManager installed, no NetPlan, and some stuff in /etc/network/interfaces. Unfortunately, since NetworkManager controls the dhcp-client, the /etc/network/interfaces interface was not obtaining an address.

The solution was to edit /etc/NetworkManager/conf.d/10-ignore-interfaces.conf, and comment the following line:

[keyfile] #unmanaged-devices=interface-name:eth*,interface-name:wan*,interface-name:lan*,interface-name:br*

After 'sudo service NetworkManager restart', this solves the error of NetworkManager not being able to manage strictly unmanaged interfaces, which for the espressobin, are lan0, lan1, wan.

Some commands for NetworkManager:

• nmcli
• nmcli device show
• nmcli connection show

File locations:

• /etc/NetworkManager/system-connections/ - interface configurations
• /usr/share/doc/network-manager/README.Debian - notes about managed/unmanaged devices

Some references:

• Debian NetworkManager - mostly desktop?
• Debian NetworkConfiguration - nostly server?
• NetworkManager homepage
• nmcli - command-line tool for controlling NetworkManager
• NetworkManager.conf - NetworkManager configuration file
• NetworkManager - archlinux view of NetworkManager, with a section on VPN connectivity based upon profiles

Obtaining interface list, which can be used to drill down into details:

$ ls /sys/class/net/
br0  eno1  enp5s0  lo  lxcbr0  veth-tf64-v90  vlan90  vlan90_br0  wlp6s0
rpb@nuc:~/data/passwords$ ls /sys/class/net/enp5s0
addr_assign_type  carrier             dev_id    gro_flush_timeout  master                operstate       proto_down  testing       upper_br0
address           carrier_changes     dev_port  ifalias            mtu                   phys_port_id    queues      threaded
addr_len          carrier_down_count  dormant   ifindex            name_assign_type      phys_port_name  speed       tx_queue_len
broadcast         carrier_up_count    duplex    iflink             napi_defer_hard_irqs  phys_switch_id  statistics  type
brport            device              flags     link_mode          netdev_group          power           subsystem   uevent

Provides some descriptive details and places of interest:

$ sudo udevadm test-builtin net_id /sys/class/net/enp5s0
Trying to open "/etc/systemd/hwdb/hwdb.bin"...
Trying to open "/etc/udev/hwdb.bin"...
Trying to open "/usr/lib/systemd/hwdb/hwdb.bin"...
Trying to open "/lib/systemd/hwdb/hwdb.bin"...
Trying to open "/lib/udev/hwdb.bin"...
=== trie on-disk ===
tool version:          252
file size:        12198286 bytes
header size             80 bytes
strings            2478998 bytes
nodes              9719208 bytes
Loading kernel module index.
Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy
Found container virtualization none.
Using default interface naming scheme 'v252'.
Parsed configuration file "/usr/lib/systemd/network/99-default.link"
Parsed configuration file "/usr/lib/systemd/network/73-usb-net-by-mac.link"
Created link configuration context.
ID_NET_NAMING_SCHEME=v252
ID_NET_NAME_MAC=enx54b2030473fa
enp5s0: MAC address identifier: hw_addr=54:b2:03:04:73:fa → x54b2030473fa
ID_OUI_FROM_DATABASE=PEGATRON CORPORATION
sd-device: Failed to chase symlinks in "/sys/devices/pci0000:00/0000:00:1c.1/0000:05:00.0/of_node".
sd-device: Failed to chase symlinks in "/sys/devices/pci0000:00/0000:00:1c.1/0000:05:00.0/physfn".
enp5s0: Parsing slot information from PCI device sysname "0000:05:00.0": success
enp5s0: dev_port=0
enp5s0: PCI path identifier: domain=0 bus=5 slot=0 func=0 phys_port= dev_port=0 → p5s0
ID_NET_NAME_PATH=enp5s0
Unload kernel module index.
Unloaded link configuration context.

The raw details contributing to the previous data:

$ udevadm info /sys/class/net/enp5s0
P: /devices/pci0000:00/0000:00:1c.1/0000:05:00.0/net/enp5s0
M: enp5s0
R: 0
U: net
I: 2
E: DEVPATH=/devices/pci0000:00/0000:00:1c.1/0000:05:00.0/net/enp5s0
E: SUBSYSTEM=net
E: INTERFACE=enp5s0
E: IFINDEX=2
E: USEC_INITIALIZED=5978173
E: ID_NET_NAMING_SCHEME=v252
E: ID_NET_NAME_MAC=enx54b2030473fa
E: ID_OUI_FROM_DATABASE=PEGATRON CORPORATION
E: ID_NET_NAME_PATH=enp5s0
E: ID_BUS=pci
E: ID_VENDOR_ID=0x8086
E: ID_MODEL_ID=0x157b
E: ID_PCI_CLASS_FROM_DATABASE=Network controller
E: ID_PCI_SUBCLASS_FROM_DATABASE=Ethernet controller
E: ID_VENDOR_FROM_DATABASE=Intel Corporation
E: ID_MODEL_FROM_DATABASE=I210 Gigabit Network Connection
E: ID_PATH=pci-0000:05:00.0
E: ID_PATH_TAG=pci-0000_05_00_0
E: ID_NET_DRIVER=igb
E: ID_NET_LINK_FILE=/usr/lib/systemd/network/99-default.link
E: ID_NET_NAME=enp5s0
E: SYSTEMD_ALIAS=/sys/subsystem/net/devices/enp5s0
E: TAGS=:systemd:
E: CURRENT_TAGS=:systemd:

Load balancing is prevalent in practical application (e.g., web) deployments seen today. One such load balancer, HAProxy, remains relevant as an open-source, easy-to-use system. In the context of web systems, the load balancer tier possesses significant influence over system performance and the incurred cost, which is decisive for cloud-based deployments. Therefore, it is imperative to properly tune the load balancer configuration and get the most performance out of the existing resources. In this technical report, we first introduce the HAProxy architecture and its load balancing methods. Then, we discuss fine-tuning parameters within this load balancer and examine their performances in face of various workload intensities. Our evaluation encompasses various types of web requests and homogeneous and heterogeneous back-ends. Lastly, based on the findings of this study, we present a set of best practices to optimally configure HAProxy.

Load Balancer Tuning: Comparative Analysis of HAProxy Load Balancing Methods

using ip link in brief mode:

# ip -br link
lo               UNKNOWN        00:00:00:00:00:00 
eno1             UP             c8:1f:66:ea:43:20 
eno2             UP             c8:1f:66:ea:43:21 
eno3             UP             c8:1f:66:ea:43:22 
eno4             DOWN           c8:1f:66:ea:43:23 

using ip addr in brief mode:

# ip -br a
lo               UNKNOWN        127.0.0.1/8 10.32.0.12/32 ::1/128
eno1             UP
eno2             UP
eno3             UP             10.32.3.12/24 fe80::ca1f:66ff:feea:4322/64
eno4             DOWN

Access to the SFP style:

# ethtool -m enp65s0f0np0
Identifier                                : 0x03 (SFP)
Extended identifier                       : 0x04 (GBIC/SFP defined by 2-wire interface ID)
Connector                                 : 0x23 (No separable connector)
Transceiver codes                         : 0x01 0x00 0x00 0x04 0x00 0x04 0x00 0x00 0x0d
Transceiver type                          : Infiniband: 1X Copper Passive
Transceiver type                          : Ethernet: 1000BASE-CX
Transceiver type                          : Passive Cable
Transceiver type                          : Extended: 25G Base-CR CA-N
Encoding                                  : 0x06 (64B/66B)
BR, Nominal                               : 25750MBd
Rate identifier                           : 0x00 (unspecified)
Length (SMF,km)                           : 0km
Length (SMF)                              : 0m
Length (50um)                             : 0m
Length (62.5um)                           : 0m
Length (Copper)                           : 1m
Length (OM3)                              : 0m
Passive Cu cmplnce.                       : 0x01 (SFF-8431 appendix E) [SFF-8472 rev10.4 only]
Vendor name                               : Mellanox
Vendor OUI                                : 00:02:c9
Vendor PN                                 : MCP2M00-A001E30N
Vendor rev                                : A4
Option values                             : 0x00 0x00
BR margin, max                            : 0%
BR margin, min                            : 0%
Vendor SN                                 : MT2010VB00194
Date code                                 : 200219
Optical diagnostics support               : No

driver and firmware info:

# ethtool -i enp65s0f0np0
driver: mlx5_core
version: 5.15.39-4-pve
firmware-version: 16.34.1002 (MT_0000000183)
expansion-rom-version:
bus-info: 0000:41:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes

The dummy interface is not used in the actual routing process in most cases. The dummy ip address is inserted into the local route table as a local address (check it out yourself: ip ro sh table local). Ingress packets from all interfaces are checked against this table first (as it is the maximum priority rule in the ip rule), and if a local-type route matches, the packet is queued for local delivery. Egress packets are even simpler: your default route will point towards your physical interface; the packets will be queued directly to that interface's queues and you'll keep all your regular GSOs and GROs -- at least the ones that are managed by the kernel.

There are standard ways many nics can offload packets: TSO (tcp segment offloading), gso (generic segment offloading), gro (generic receive offload), lro (large receive offload) are the common ones. You can see which ones are enabled with ethtool -k interface_name, interface usually has to be down to change them.

From the P4-dev mailing list, an interesting tool which deep dives wifi, P4, the kernel, ...

you may want to look at the BATMAN and P4 examples in Mininet-WiFi: Manet Routing Protocols or P4 Programming Protocol-Independent Packet Processors .

BTW, we recently fully open-sourced here: mn-wifi-ebook.

The English version of the Mininet-WiFi book: The Mininet-WiFi Book

Found on a email list:

The Netgear GS108T is my typical go-to "not a dumb switch". 8 ports for about $80.

Make sure you get the v3 if you want most of the modern IPv6 L2 features (you also get some very limited L3 capabilities).

Extra bonus with the GS108Tv3, and anything else based on the RTL8380, is that you can run OpenWrt on it.

   [ Noah Meyerhans ]
   * net: Use fq_codel as the default network qdisc (Closes: #890343)

From ticket #890343:

by default Debian uses the pfifo_fast network queuing algorithm:

# tc -s qdisc show
[...]
qdisc pfifo_fast 0: dev eth0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 413728102 bytes 475015 pkt (dropped 0, overlimits 0 requeues 785)
 backlog 0b 0p requeues 785

The systemd source package contains this file:

  ./systemd-237/sysctl.d/50-default.conf:net.core.default_qdisc = fq_codel

whose purpose is to set the default queuing algorithm to fq_codel.  
According to the NEWS file, this is a better alternative:

        * The default sysctl.d/ snippets will now set:

                net.core.default_qdisc = fq_codel

          This selects Fair Queuing Controlled Delay as the default
          queuing discipline for network interfaces. fq_codel helps
          fight the network bufferbloat problem. It is believed to be
          a good default with no tuning required for most workloads.
          Downstream distributions may override this choice. On 10Gbit
          servers that do not do forwarding, "fq" may perform better.
          Systems without a good clocksource should use "pfifo_fast".

However the 50-default.conf file is not in the Debian binary package.
Is this intentional or an omission?
Could it be possible to enable fq_codel by default?

Flexible routing has now come down in price with the Raspberry Pi Compute Module 4 IoT Router Carrier Board Mini which takes a Raspberry Pi Compute Module 4 and places it on a Compute Module 4 IoT Router Carrier Board Mini.

Here is someone building a Raspberry Pi Firewall and Router with DF Robot Dual NIC with Ubuntu.

How to get stats at all of the different points in the stack to track down reasons for dropped packets:

• ethtool -S for h/w and driver
• tc -s for drops by the qdisc
• /proc/net/softnet_stat for drops at the backlog layer
• netstat -s for network and transport layer
• ip -s

The key is the 'ether' specifier. The -e option shows the mac address in the output:

$ sudo tcpdump -n -i vlan50 ether host e4:54:e8:29:44:2d or ether host ff:ff:ff:ff:ff:ff -vv -e

The country code is not read from ACPI but from the EEPROM of the WiFi-card (0x0 "World Regulatory Domain" by default).

How to Build your Own Wireless Router (Part 3) - kernel requires code patch to enable 5GHz operations.

ath10k locks to regulatory domain US on ACPI platforms

From Phoronix - Linux 5.6 To Bring FQ-PIE Packet Scheduler To Help Fight Bufferbloat, there are some example configurations for using different buffering mechanisms (Implementing the Flow Queue PIE AQM in the Linux Kernel).

For wireless clients:

net.core.default_qdisc=fq
net.ipv4.tcp_congestion_control = bbr

and fq_codel (codel) is better suited for router.

Another possibility:

I had pretty amazing results with cake. I prefer it over fq_codel and it is now part of main line since 4.19.

Cake - Common Applications Kept Enhanced

In the past the internet would just grind to a halt when the Apple devices started cloud syncing. They would totally swamp the outgoing bandwidth. Now nobody notices when an Apple device is syncing.

Open firmware for small routers for congestion management:

The OpenWrt issue with 4MB has to do with using newer kernels. For those devices, DD-WRT is a better bet (still on kernel 3.10).