Linux

During the early days of Debian Linux, one could get away with a 100mb boot partition. With the explosion of included firmware files in the initramfs file, the boot directory needs to be 500mb or even 1gb in size. I have a couple older machines I have not yet rebuilt, which have limited size. I resort to copying various initrd.img files in and out as I upgrade kernels or boot into older versions of the kernel.

Sometimes I have forgotten to properly copy an image back into /boot and run update-grub. When I do that, I have to boot into grub and run the grub command line. Some commands that I run are as follows:

Set a variable so when requesting help, you can page through entries:

grub> set pager=1

List the various mount points:

grub> ls

List the files in a particular mount point

grub> ls (hd0,msdos2)/root

Startup commands, depending upon where linux and initrd files are found:

grub> set root=(hd0,msdos2)
grub> linux (hd0,msdos1)/vmlinux-6.1.0-15-amd64 root=LABEL=main
grub> initrd (hd0,msdos2)/root/initramfs/initrd.img-6.1.0-15-amd64
grub> boot

# dd if=Downloads/iso/debian-testing-amd64-netinst.iso of=/dev/sda bs=1M status=progress conv=fdatasync

'fdatasync' is equivalent to running 'sync' as a second command

watch kern.log to determine on which drive the USB is mounted when inserted

ensure that the USB is not auto-mounted by any other application or service

$ ucfq /etc/ssh/sshd_config
Configuration file                            Package             Exists Changed
/etc/ssh/sshd_config                          openssh-server      Yes     No

• Linux Virtual Server - a highly scalable and highly available server built on a cluster of real servers, with the load balancer running on the Linux operating system (last date on the web page: 2012)
• Linux-VServer - provides virtualization for GNU/Linux systems. This is accomplished by kernel level isolation. It allows to run multiple virtual units at once. Those units are sufficiently isolated to guarantee the required security, but utilize available resources efficiently, as they run on the same kernel. (a precursor to LXC) (last mod 2018)

ls -srSk ~/Downloads/

• s - size in blocks
• r - reverse order (largest at bottom)
• k - 1024 byte block size
• S - sort by size

ls -d ~/Downloads/*

• Shows path to all files in the directory

According to the slides from a 2023 Linux Storage, Filesystem, Memory-Management and BPF Summit talk, guests operating through the netkit device (which was called "meta" at that time) are able to attain TCP data-transmission rates that are just as high as can be had by running directly on the host. The performance penalty for running within a guest has, in other words, been entirely removed.

• Learn Nokia SR Linux - Service Router Linux
• SR tutorial with containerlab
• Microsoft has crafted a switch OS on Debian Linux

I found out the hard way that SaltStack and Debian no longer place nice together. I had upgraded a Debian installation from Bullseye to Bookworm, along with the resident Salt Minion. When attempting to use the minion, it no longer starts up, due to various imports no longer working. Which was due to the salt-minion not being upgraded. The error message would started this odyssey:

salt ImportError: cannot import name 'Markup' from 'jinja2'

Taking a look at the Debian Developer Information for Salt, the last version started in 'unstable' was 3004.1 back in December of 2022. This is now almost 8 months later and little or no movement. There was some mention in a ticket somewhere that Salt release cycles don't cater to Debian stable release cycles. Not sure if that is a legitimate reason or not, but, well, for whatever reason, SaltStack management in Debian is no longer a simple no brainer.

However, after a little digging, there is a way to run SaltStack versions 3006 (current as of this writing). It is simple to install on Bullseye, but not easily done on Bookworm.

On Bullseye (as root, or implies sudo):

# cd ~ # apt remove salt-minion salt-master # apt install curl # curl -L https://bootstrap.saltstack.com -o install_salt.sh # sh install_salt.sh -M onedir

The '-M' installs the salt master at the same time (for machines running master). If you forget to do that, you'll need to diagnose and fix the systemctl mask error with the following:

# apt install file # file /etc/systemd/system/salt-master.service # rm /etc/systemd/system/salt-master.service # systemctl daemon-reload # sh install_salt.sh -M onedir

The 'sh install_salt.sh -M onedir' should show a symlink to /dev/nul, which the 'rm ...' will fix.

On Bookworm, the bootstrap isn't scheduled to work till beginning of 2024 sometime I think with Salt 3007 or 3008 -- more info in [FEATURE REQUEST] Add Salt support for Debian 12 #64223 .

In the meantime, I had to cheat a bit:

• in /etc/debian_version, change 12.0 to 11.0
• in /etc/apt/sources.list, change bookworm to bullseye
• rm /etc/apt/sources.list.d/salt.list
• run apt update
• run the commands listed above for installing the one or both the salt services
• restore /etc/debian_version and /etc/apt/sources.list to their original content

I'm sure there are more elegant ways of doing this, but this worked to fake the needed version 11 in the installation script and directory traversal requirements

Note, more info on the Salt Install/Bootstrap Process.

I had an application running which had an open file it was actively using.

I accidentally performed an rm (remove) on the file rather than another I actually meant to remove.

Due to Linux's method of linking files, even though the directory entry link to the file was removed, the file is still open and has an additional link via the application process pseudo directory.

The process sub-directory is composed of the process id. The process id can be found with something like:

$ pidof BasketTrading
2663042

The deleted file can then be found with:

$ lsof -p 2663042 | grep deleted
BasketTra 2663042  rpb   18u      REG               0,46   2739200 29438986 /home/.../BasketTrading.db (deleted)

The 18u reflects the file descriptor used for the file. This can be used to perform a simple copy of the file to an alternate location. A link does not seem to fix it. You may want to complete any outstanding writes to the file first. But do not close it or the application. If you do, the file will be unrecoverable.

$ ls /proc/2663042/fd/18
/proc/2663042/fd/18

$ cp /proc/2663042/fd/18 /home/.../BasketTrading.db.rescue

Here is an interesting note from Connor's Blog - an automatic reboot after kernel panic with the following sysctl setting:

$ sysctl -w kernel.panic=60

... or on the kernel command line:

panic=60

Yes, this is a very old device (back to 2017 or so). But I have a couple, and I need to keep a few reference notes to see if it is upgradeable.

• Linux for Tegra R21.8 found from Jetson Linux Archive -- shows a 3.10.40 version kernel with a 32 bit driver package -- pretty old, if only 32 bit.
• Installing Debian on nvidia Jetson TK1 - based on the Tegra K1 chip (also known as Tegra 124). The Tegra K1 (codenamed "Logan") features a quad-core 32-bit ARM Cortex-A15 CPU and Kepler GPU (GK20A) with 192 CUDA cores. The Jetson TK1 can run Debian's armhf port.

References

• systemd-networkd - from archlinux, with reference to systemd-nspawn for container networking. -- interesting note: "it is possible to run Docker containers inside an unprivileged systemd-nspawn container with cgroups v2 enabled". But from an old talk:
  • Creating containers with systemd-nspawn - It is targeted at "building, testing, debugging, and profiling", not at deployment. systemd-nspawn uses the same kernel APIs that the other two tools use, but is not a competitor to them because it is not targeted at running in a production environment.

Gone are the good 'ole days of using /etc/network/interfaces to manage basic networking stack configurations. There seems to be an explosion of alternate ways, each stepping on each other's toes: /etc/network/interfaces, NetworkManager, NetPlan, systemd-networking, ...

A problem I had the other day was where a new installation of an espressobin had NetworkManager installed, no NetPlan, and some stuff in /etc/network/interfaces. Unfortunately, since NetworkManager controls the dhcp-client, the /etc/network/interfaces interface was not obtaining an address.

The solution was to edit /etc/NetworkManager/conf.d/10-ignore-interfaces.conf, and comment the following line:

[keyfile] #unmanaged-devices=interface-name:eth*,interface-name:wan*,interface-name:lan*,interface-name:br*

After 'sudo service NetworkManager restart', this solves the error of NetworkManager not being able to manage strictly unmanaged interfaces, which for the espressobin, are lan0, lan1, wan.

Some commands for NetworkManager:

• nmcli
• nmcli device show
• nmcli connection show

File locations:

• /etc/NetworkManager/system-connections/ - interface configurations
• /usr/share/doc/network-manager/README.Debian - notes about managed/unmanaged devices

Some references:

• Debian NetworkManager - mostly desktop?
• Debian NetworkConfiguration - nostly server?
• NetworkManager homepage
• nmcli - command-line tool for controlling NetworkManager
• NetworkManager.conf - NetworkManager configuration file
• NetworkManager - archlinux view of NetworkManager, with a section on VPN connectivity based upon profiles

It is always interesting looking at bug reports. They uncover interesting nooks and crannies of system tooling. In reading Debian Bug#1037194, there is a command for watching dbus:

$ sudo dbus-monitor --system

To keep locally generated systemd service files separate from distribution files (as an example):

$ sudo mkdir -p /usr/local/lib/systemd/system
$ sudo vim /usr/local/lib/systemd/system/bme680.service
$ sudo systemctl enable bme680
$ sudo systemctl daemon-reload
$ sudo systemctl start bme680

An example systemd service file in /etc/systemd/system might look like:

[Unit]
Description=BME680 Collector
Documentation=https://github.com/rburkholder/bme680
After=network.target

[Service]
Type=simple
User=debian
WorkingDirectory=/home/debian
Environment="ID=02" "LOCATION=top floor"
ExecStart=/home/debian/bme680/build/bme680-mqtt ${ID} ${LOCATION}
ExecReload=kill -HUP $MAINPID
KillMode=process
Restart=on-failure
RestartSec=5s
StandardOutput=null
# note change with logrotate implemented, or to a memory file

[Install]
WantedBy=multi-user.target

The example from ecowitt2mqtt got me started - is a small CLI/web server that can receive data from Fine Offset weather stations (and their numerous white-labeled counterparts, like Ecowitt and Ambient Weather), adjust that data in numerous ways, and send it on to one or more MQTT brokers.

Reference to:

• systemd.service — Service unit configuration
• systemd.exec — Execution environment configuration - has list of expanded exit codes