Raymond P. Burkholder - Things I Do

Lifted from Installing Virtio Drivers for Windows on Proxmox to Enable Paravirtualization / Fix Incorrect Memory Monitoring, there is a section at the end which discusses settings for disk caching modes in Proxmox. The terms relate to how frequently data is flushed to the underlying storage system. There are trade-offs for performance and durability.

• Direct sync - highest data security but has poor performance - ensures that every time data is written to memory, it is immediately synchronized to the disk. This means each write operation introduces disk latency because the system must wait for the disk to confirm the write is complete before continuing with other operations. Although this method provides the highest data security, frequent disk operations lead to lower performance.
• Write through - balance between data consistency and performance - writing data to memory and immediately writes it to disk without waiting for disk confirmation. It offers good data consistency and high performance by reducing the time spent waiting for disk confirmation. However, disk write operations still occur, leading to some performance overhead.
• Write back - improves performance but comes with certain data risks - first writes data to the memory cache and writes it back to the disk at an appropriate time. This reduces the number of disk accesses and improves system performance. However, because data is cached in memory, there is a risk of data loss if the system crashes or experiences a power outage before writing back to the disk.
• Write back (unsafe) - highest performance (fastest among all modes) but compromises data security and consistency - similar to the regular write back method, but it makes no guarantees about data consistency. This means that in the event of a system failure or power outage, data not yet written back to the disk could be permanently lost. While this method provides higher performance, it sacrifices data security and consistency. RamFS would be a use case here.

Perform your own testing given your own physical infrastructure and weigh the pro's and con's.

Starting with version VE 8.4, proxmox host directories can be shared into a windows or linux virtual machine. I have a version 9 running here as an example:

# pveversion
pve-manager/9.0.6/49c767b70aeb6648 (running kernel: 6.14.8-2-pve)

Check to see that the appropriate package is installed, it should be by default:

# dpkg -l | grep virtiofsd
ii  virtiofsd         1.13.2-1      amd64     Virtio-fs vhost-user device daemon

Create a directory for sharing, use something like /mnt or /var/local, depending upon your preference:

# mkdir /var/local/shared

In the proxmox management site, navigate Datacenter -> Directory Mappings, then add a mapping with the directory just created. The name supplied will be the name seen in the virtual machine.

Also in the proxmox management site, navigate to the virtual machine settings -> Hardware and add a Virtiofs Filesystem Passthrough. Select the directory id you just created. Select the 'Direct IO' box (may be optional). Then restart the machine if already started to have the setting take effect.

In a Windows virtual machine, be sure you've installed the Windows VirtIO Drivers.

Open the Services cmdlet (services.msc). There should be a 'VirtIO-FS Service". Startup type can be set to automatic. If only temporary use is required, simply start the service. A new device should become visible in This PC in file explorer.

Some sites indicate that WinFSP is required. It is mentioned at How to install virtiofs drivers on Windows, but the virtio drivers may already supplied with the initial VirtIO driver install.

Risk Manager for Intrusion Tolerant Systems: Enhancing HAL 9000 with New Scoring and Data Sources

Intrusion Tolerant Systems (ITSs) have become increasingly critical due to the rise of multi-domain adversaries exploiting diverse attack surfaces. ITS architectures aim to tolerate intrusions, ensuring system compromise is prevented or mitigated even with adversary presence. Existing ITS solutions often employ Risk Managers leveraging public security intelligence to adjust system defenses dynamically against emerging threats. However, these approaches rely heavily on databases like NVD and ExploitDB, which require manual analysis for newly discovered vulnerabilities. This dependency limits the system's responsiveness to rapidly evolving threats. HAL 9000, an ITS Risk Manager introduced in our prior work, addressed these challenges through machine learning. By analyzing descriptions of known vulnerabilities, HAL 9000 predicts and assesses new vulnerabilities automatically. To calculate the risk of a system, it also incorporates the Exploitability Probability Scoring system to estimate the likelihood of exploitation within 30 days, enhancing proactive defense capabilities.

Despite its success, HAL 9000's reliance on NVD and ExploitDB knowledge is a limitation, considering the availability of other sources of information. This extended work introduces a custom-built scraper that continuously mines diverse threat sources, including security advisories, research forums, and real-time exploit proofs-of-concept. This significantly expands HAL 9000's intelligence base, enabling earlier detection and assessment of unverified vulnerabilities. Our evaluation demonstrates that integrating scraper-derived intelligence with HAL 9000's risk management framework substantially improves its ability to address emerging threats. This paper details the scraper's integration into the architecture, its role in providing additional information on new threats, and the effects on HAL 9000's management.

This is an interesting take on security: toleration of intruders to systems. I suppose this is somewhat akin to limiting blast radius. One interesting nugget:

... system applies a proactive recovery mechanism to protect the system from undetected attacks. A central controller periodically rotates the state of each VM, transitioning between active, cleansing, ready, and active states.

This would be the pre-emption step of what file checksum utilities would do. However, this doesn't eliminate the types of intrusions where intruders make use of elevated account access.

AI agentic programming is an emerging paradigm in which large language models (LLMs) autonomously plan, execute, and interact with external tools like compilers, debuggers, and version control systems to iteratively perform complex software development tasks. Unlike conventional code generation tools, agentic systems are capable of decomposing high-level goals, coordinating multi-step processes, and adapting their behavior based on intermediate feedback. These capabilities are transforming the software development practice. As this emerging field evolves rapidly, there is a need to define its scope, consolidate its technical foundations, and identify open research challenges. This survey provides a comprehensive and timely review of AI agentic programming. We introduce a taxonomy of agent behaviors and system architectures, and examine core techniques including planning, memory and context management, tool integration, and execution monitoring. We also analyze existing benchmarks and evaluation methodologies used to assess coding agent performance. Our study identifies several key challenges, including limitations in handling long context, a lack of persistent memory across tasks, and concerns around safety, alignment with user intent, and collaboration with human developers. We discuss emerging opportunities to improve the reliability, adaptability, and transparency of agentic systems. By synthesizing recent advances and outlining future directions, this survey aims to provide a foundation for research and development in building the next generation of intelligent and trustworthy AI coding agents.

Open Questions about Time and Self-reference in Living Systems

Living systems exhibit a range of fundamental characteristics: they are active, self-referential, self-modifying systems. This paper explores how these characteristics create challenges for conventional scientific approaches and why they require new theoretical and formal frameworks. We introduce a distinction between 'natural time', the continuing present of physical processes, and 'representational time', with its framework of past, present and future that emerges with life itself. Representational time enables memory, learning and prediction, functions of living systems essential for their survival. Through examples from evolution, embryogenesis and metamorphosis we show how living systems navigate the apparent contradictions arising from self-reference as natural time unwinds self-referential loops into developmental spirals. Conventional mathematical and computational formalisms struggle to model self-referential and self-modifying systems without running into paradox. We identify promising new directions for modelling self-referential systems, including domain theory, co-algebra, genetic programming, and self-modifying algorithms. There are broad implications for biology, cognitive science and social sciences, because self-reference and self-modification are not problems to be avoided but core features of living systems that must be modelled to understand life's open-ended creativity.

I've done some pre-seeding to be able to auto-install Debian on bare metal and as virtual machines. Using classes is a useful addition for conditional execution of configuration management.

What you probably ought to be doing instead is specifying the option you want via the `classes` setting (an alias for `auto-install/classes`), and then implementing the behaviour you want by setting `preseed/run` in a minimal preseed.cfg, and having a script do whatever you want in response to the classes= setting you specified. So your preseed.cfg would just be:

d-i	preseed/run	string start.sh

and your start.sh would be something like:

#!/bin/sh
. /usr/share/debconf/confmodule

db_get auto-install/classes
db_set preseed/include "${RET:-default}".cfg;;

(you might want to check that the value of $RET makes sense before using it)

Then the real configs go in `default.cfg` and `alternative.cfg` (assuming you're planning on specifying `classes=alternative`)

You may be able to get inspiration from here: https://hands.com/d-i/

although the class handling there is quite deeply buried.

Here's a simpler example of using preseed/run to do odd things to D-I, that while not involving classes, is easier to follow: https://hands.com/d-i/bug/846002/

BTW `auto-install/classes` is reserved for users (it is not used by D-I internally), so it's there specifically for you to implement whatever scheme you like.

Additional information on pre-seeding can be found at B.2. Using preseeding.

A reference for my future self for some interesting troubleshooting capability

When triggering the issue please issue as well a Alt+SysRq+t to dump a list of current tasks and their information to the log. (on systems where you do not have the SysRq key, it might be the PrintScreen one). You need to enable as well debugging dumps, so have to enable it:

echo 8 > /proc/sys/kernel/sysrq

then once you have triggered the crash issue a Alt+SysRq+t and provide the log got over the netconsole.

More information on SysRq key hacks: https://docs.kernel.org/admin-guide/sysrq.html

Lastly, since you have a efi system, we might take as well advantage of logging via pstore. It seems active on your system, so check on next crash please as well /var/lib/systemd/pstore directory for fresh crash logs.

For pstore: https://docs.kernel.org/admin-guide/pstore-blk.html and systemd-pstore.service(8).

... so let's go the bisect route, I will replicate the excellent small howto from Uwe Kleine-Koenig here.

The whole will involve to compile multiple kernels. First we need to prepare a config for your system and to prepare the respective upstream versions, in your case we want to bisect the stable versions in one specific branch, so we can shortcut, and we know we want to bisect changes between 6.12.35 and 6.12.38 in your case.

    git clone --single-branch -b linux-6.12.y https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
    cd linux-stable
    git checkout v6.12.35
    cp /boot/config-$(uname -r) .config
    yes '' | make localmodconfig
    make savedefconfig
    mv defconfig arch/x86/configs/my_defconfig

As a fist step now compile a local 6.12.35 to ensure it is "good".

    # test 6.12.35 to ensure this is "good"
    make my_defconfig
    make -j $(nproc) bindeb-pkg
    ... install the resulting .deb package and confirm the problem is not present.

And now the same for the version you suspect is broken, is "bad" for git bisect speach.

    # test 6.12.38 to ensure this is "bad"
    git checkout v6.12.38
    make my_defconfig
    make -j $(nproc) bindeb-pkg
    ... install the resulting .deb package and confirm the problem with audio exists.

So we have now a range of from good to bad kernel and we can start the bisect:

    git bisect start v6.12.38 v6.12.35

    make my_defconfig
    make -j $(nproc) bindeb-pkg
    ... install, try to boot and verify the state of the problem

and if the problem is hit run:

    git bisect bad

    git bisect good

Please pay attention to always select the just built kernel for booting, it won't always be the default kernel picked up by grub.

Then provide the output of

    git bisect log

In the course of the bisection you might have to uninstall previous kernels again to not exhaust the disk space in /boot. Also in the end uninstall all self-built kernels again.

Learned a new detail command for USB port details.

The below is a listing from an Asus X670e motherboard. Notice that the command provides port speeds (which implies a USB port type, which may be dependent upon capability/type of device connected). And yes, I still have a Microsoft 4000 keyboard. I wish they still made them. Extremely comfortable and has an excellent layout. Any good places to find them?

$ lsusb -vt
/:  Bus 001.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/2p, 480M
    ID 1d6b:0002 Linux Foundation 2.0 root hub
/:  Bus 002.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/2p, 10000M
    ID 1d6b:0003 Linux Foundation 3.0 root hub
/:  Bus 003.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/12p, 480M
    ID 1d6b:0002 Linux Foundation 2.0 root hub
    |__ Port 002: Dev 002, If 0, Class=Audio, Driver=snd-usb-audio, 480M
        ID 0424:3fb7 Microchip Technology, Inc. (formerly SMSC)
    |__ Port 002: Dev 002, If 1, Class=Audio, Driver=snd-usb-audio, 480M
        ID 0424:3fb7 Microchip Technology, Inc. (formerly SMSC)
    |__ Port 002: Dev 002, If 2, Class=Audio, Driver=snd-usb-audio, 480M
        ID 0424:3fb7 Microchip Technology, Inc. (formerly SMSC)
    |__ Port 002: Dev 002, If 3, Class=Audio, Driver=snd-usb-audio, 480M
        ID 0424:3fb7 Microchip Technology, Inc. (formerly SMSC)
    |__ Port 006: Dev 003, If 0, Class=Wireless, Driver=btusb, 480M
        ID 0489:e0e2 Foxconn / Hon Hai
    |__ Port 006: Dev 003, If 1, Class=Wireless, Driver=btusb, 480M
        ID 0489:e0e2 Foxconn / Hon Hai
    |__ Port 006: Dev 003, If 2, Class=Wireless, Driver=btusb, 480M
        ID 0489:e0e2 Foxconn / Hon Hai
    |__ Port 007: Dev 004, If 0, Class=Vendor Specific Class, Driver=[none], 12M
        ID 0b05:19af ASUSTek Computer, Inc.
    |__ Port 007: Dev 004, If 2, Class=Human Interface Device, Driver=usbhid, 12M
        ID 0b05:19af ASUSTek Computer, Inc.
/:  Bus 004.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/5p, 20000M/x2
    ID 1d6b:0003 Linux Foundation 3.0 root hub
/:  Bus 005.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/12p, 480M
    ID 1d6b:0002 Linux Foundation 2.0 root hub
/:  Bus 006.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/5p, 20000M/x2
    ID 1d6b:0003 Linux Foundation 3.0 root hub
/:  Bus 007.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/2p, 480M
    ID 1d6b:0002 Linux Foundation 2.0 root hub
/:  Bus 008.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/2p, 10000M
    ID 1d6b:0003 Linux Foundation 3.0 root hub
/:  Bus 009.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/2p, 480M
    ID 1d6b:0002 Linux Foundation 2.0 root hub
    |__ Port 002: Dev 003, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
        ID 045e:00db Microsoft Corp. Natural Ergonomic Keyboard 4000 V1.0
    |__ Port 002: Dev 003, If 1, Class=Human Interface Device, Driver=usbhid, 1.5M

I needed to step back a few commits to check where something went wrong. Or at what point something went wrong.

This works with a mixture of local commits and with primary commits sent to remote.

• git stash # whatever isn't committed yet
• git checkout X #go back in time - where X is the SHA256 hash of the commit
• git checkout . # throw away any changes
• git checkout master #go forward to current
• git stash pop # bring back any unsaved changes

A Survey of AIOps in the Era of Large Language Models - the paper goes into some detail of challenges of AIOps, and by association, relates the difficulty of vibe-fixing our way out.

As large language models (LLMs) grow increasingly sophisticated and pervasive, their application to various Artificial Intelligence for IT Operations (AIOps) tasks has garnered significant attention. However, a comprehensive understanding of the impact, potential, and limitations of LLMs in AIOps remains in its infancy. To address this gap, we conducted a detailed survey of LLM4AIOps, focusing on how LLMs can optimize processes and improve outcomes in this domain. We analyzed 183 research papers published between January 2020 and December 2024 to answer four key research questions (RQs). In RQ1, we examine the diverse failure data sources utilized, including advanced LLM-based processing techniques for legacy data and the incorporation of new data sources enabled by LLMs. RQ2 explores the evolution of AIOps tasks, highlighting the emergence of novel tasks and the publication trends across these tasks. RQ3 investigates the various LLM-based methods applied to address AIOps challenges. Finally, RQ4 reviews evaluation methodologies tailored to assess LLM-integrated AIOps approaches. Based on our findings, we discuss the state-of-the-art advancements and trends, identify gaps in existing research, and propose promising directions for future exploration.

Machine Bullshit: Characterizing the Emergent Disregard for Truth in Large Language Models - a mathematical rendering of a Bullshit Index

Bullshit, as conceptualized by philosopher Harry Frankfurt, refers to statements made without regard to their truth value. While previous work has explored large language model (LLM) hallucination and sycophancy, we propose machine bullshit as an overarching conceptual framework that can allow researchers to characterize the broader phenomenon of emergent loss of truthfulness in LLMs and shed light on its underlying mechanisms. We introduce the Bullshit Index, a novel metric quantifying LLMs' indifference to truth, and propose a complementary taxonomy analyzing four qualitative forms of bullshit: empty rhetoric, paltering, weasel words, and unverified claims. We conduct empirical evaluations on the Marketplace dataset, the Political Neutrality dataset, and our new BullshitEval benchmark (2,400 scenarios spanning 100 AI assistants) explicitly designed to evaluate machine bullshit. Our results demonstrate that model fine-tuning with reinforcement learning from human feedback (RLHF) significantly exacerbates bullshit and inference-time chain-of-thought (CoT) prompting notably amplify specific bullshit forms, particularly empty rhetoric and paltering. We also observe prevalent machine bullshit in political contexts, with weasel words as the dominant strategy. Our findings highlight systematic challenges in AI alignment and provide new insights toward more truthful LLM behavior.

One wouldn't normally associate these together, but after encountering the pebble in the shoe for far too long, it is nice to have a 'mostly' resolution.

My first introduction to the wonders of Wayland was when I assembled my new computer based upon the Asus ProArt X670E-CREATOR WIFI motherboard (motherboard works just fine) and installed Debian Trixie with the KDE windowing system. In this install, the default rendering manager is Wayland. For a user running a single monitor and a single virtual desktop, this is probably the cat's meow.

However, having used LXDE with virtual desktops, and when ever restarting Firefox, all my multitude of FireFox windows were placed in their original positions on my multiple monitors and across my virtual desktops. Trying to do this in the world of KDE/Wayland turned out to be impossible.

My interim solution was to create a Firefox profile for each virtual desktop. At least that provided the ability to keep the windows local. It also reduced the number of windows in each profile, and accessing content was a bit snappier, as there were fewer windows. I don't know why multiple windows will slow down Firefox, but it started back in the v90's I think.

Then when I installed an NVidia RTX 5070 TI, and moved my monitors over to it, my mouse got shaky and stuttery. Some forum users suggested disconnecting and reconnecting one or more video cables may shake the problem loose. This was not a solution for me.

I was thinking that Wayland was my only option and there was no going back to X11. But yes it was possible. I simply needed to logout, choose a setting in tiny print in the lower left of the screen and select X11.

When logging in with X11, the mouse stuttering stopped.

And by happenstance, when I brought Firefox back up with all the windows, the Windows started shifting over to their original locations.

Prior to this, I didn't know: Wayland doesn’t let applications set their position, but it does allow them to set their size - so that is why I can't restore my Firefox windows to their original locations.

Wayland's positioning problem leads to a KiCad remark: KiCad's development team recently came out saying Wayland was feature poor and kind of buggy which results in their application not running well on it. Some of it due to this window placement problem.

Overall, my primary problems have disappeared, but I find that running three 4K monitors off an RTX 5070 Ti in X11 mode, everything appears a bit 'laggy'. Next weekend, I'll switch my monitors back to the on-board Video and see if the lag is more or less.

One of the reasons for moving the monitors to the NVidia card was that when monitors went into sleep mode when I was away from the computer, a few apps including a couple Wine based apps had their windows closed. Now, if I move back to the onboard video card, and retain monitor power-off, I'll see if this works.

And at the same time, maybe I can save an extra 30W that nvidia-smi says it consumes when driving my monitors.

Note, I got the NVidia card, not for gaming, but for running PyTorch/libtorch based simulations.

As a side note, there is a world of difference when running nvidia-settings when in Wayland mode and when in X11 mode.

One more reminder for myself, to use the NVidia card for monitors instead of the onboard video card, this was needed:

cat /etc/modprobe.d/nvidia.conf                                                                                                                                                                                                          
options nvidia-drm modeset=1

With the following note:

Setting modeset=1 doesn’t actually install a framebuffer console. All it really does is enable the DRIVER_MODESET capability flag in the nvidia-drm devices so that DRM clients can use the various modesetting APIs. In addition to allowing clients that talk to the low-level DRM interface to work, it’s also necessary for some PRIME-related interoperability features.

The downside, if you want to call it that, is that loading nvidia-drm with modeset=1 causes it to configure and initialize all GPUs immediately rather than waiting for a client to open the /dev/nvidia* device files. This means that some options that require a userspace application to configure them before the GPUs are initialized won’t work if they were already configured by nvidia-drm. The big example at the moment is SLI Mosaic, which is enabled by the X driver if /etc/X11/xorg.conf says it should be.

Resources:

• Wayland does work and does stuff people claim it can’t - long discussion of Wayland capabilities and non-capabilities
• Wayland + Firefox, can’t restore windows to correct location - others have had my problem with Firefox windows returning to position when using Wayland

	git clone https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
	cd linux-stable
	git checkout v6.15.4
	cp /boot/config-$(uname -r) arch/x86/configs/my_defconfig
	make my_defconfig
	make localmodconfig
	make localyesconfig
	make -j 12
	make install

From an email list:

	git clone https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
	cd linux
	cat /proc/config.gz > .config
	yes '' | make oldconfig
	make localmodconfig
	make savedefconfig
	cp defconfig arch/x86/configs/test_defconfig

	git bisect start v6.11 v6.6

Then iterate:

	make test_defconfig
	make
	make install

Test the newly installed kernel. Depending on the kernel having the problem or not run:

	git bisect good

or

	git bisect bad

Repeat until git identifed the problematic commit.

Note: Make sure to pick the right kernel to boot, it's not always the one with the highest version number. After a test you can remove the installed kernel at any time (and you probably want that to not run out of disk space).