Raymond P. Burkholder - Things I Do - Open Source

Prepping nvidia drivers for Frigate Running in Docker inside LXC on Proxmox

nospam@example.com (Raymond P. Burkholder) — Mon, 27 Apr 2026 02:34:23 +0000

Not having the desire to put frigate on a standalone server of some sort, I wanted it to run as a container in Proxmox. Since Frigate is supplied as a Docker container, and I don't want to run Docker directly on the Proxmox host, there are two choices open to me:

run Frigate in a virtual machine - prevents sharing an Nvidia card with other machines or containers, or
run Frigate in an LXC or an OCI container - an Nvidia card can be shared with multiple containers, and resource requirements are reduced, at the risk of security issues (which are reduced if the host is a private server)

I'm running on Proxmox 9.1.9 with kernel SMP PREEMPT_DYNAMIC PMX 6.17.13-4 (2026-04-21T22:03Z) x86_64 GNU/Linux

This installation is a privileged container as I had troubles with non-privileged. I'll have to come back to this based upon: Running in rootless LXC, and maybe require the user id mapping from Unprivileged LXC containers.

Some installation commands, based upon connecting Frigate to an Nvidia GPU. Some instructions come from Installing the NVIDIA Container Toolkit, which is required for connecting to the Nvidia GPU (commands assume sudo or running as root, which is default in a basic LXC container). The link contains installation, testing and troubleshooting instructions.
Continue reading "Prepping nvidia drivers for Frigate Running in Docker inside LXC on Proxmox"

LibreOffice locking up when using File/Open/Save Dialog

nospam@example.com (Raymond P. Burkholder) — Fri, 27 Mar 2026 01:02:06 +0000

Some say this happens due to Wayland usage. My usage is Wayland/X11, not that it may make a difference. It is that way because I like that fact that the windows of an application get placed back to where they were when the application is re-opened.

The solution appears to be:

Tools -> Options -> General -> Enable 'Use LibreOffice dialogs'

Anti-Spam Measures

nospam@example.com (Raymond P. Burkholder) — Sun, 27 Aug 2023 01:08:04 +0000

Request: Or people simple do not want to share own email, as it is not related for delivery, but required by the form. Thus they fill semi random email like string.

Response: That is what mailinator is for, or the temporary addresses offered by several browser plugins, or if you know they're going to use it to spam you, abuse@ their domain. nobody@example.com works as well, and 1-900-976-1212 for a "required" phone number.

General Guideline for Setting up a Mail Server

nospam@example.com (Raymond P. Burkholder) — Mon, 10 Jul 2023 22:52:53 +0000

make sure the MTA's forward and reverse DNS match
set up an SPF record, probably "v=spf1 mx ~all"
manage DNS MX, DKIM, DMARC and SPF records (SPF, DKIM, and DMARC are a recommended order of implementation)
set up DKIM signing for each domain you host, make the DKIM domain match the From: domain
start slow and look at any bounces
maybe collect DMARC stats but for a small volume MTA, not very interesting
instead of struggling with Postfix, OpenDKIM, Dovecot and friends (and losing out on quite a few features), try looking at maddy
arrange for a backup MTA
manage reverse lookup records, including managing the uncertain chain of authority between the instance and the nearest SOA
manage certificates associated with TLS for SMTP and IMAP
manage DKIM certificate
manage one's upstream to address PBL issues
keep the MTA secure and free from DOS attack
Use a static IPv4 address for your email system [counter point: If DNS is fully functional and IP addresses don't change too quickly and TTL is configured properly on DNS records, ... then why is a static IP address strictly required? ]
Do not share this IPv4 address with user machines
Do not host your email system 'in the cloud'
Make sure that your IP address is not listed in the PBL
Provide an MX record
Provide meaningful and consistent reverse DNS
Your system should say HELO (or EHLO) with its hostname
Keep your software completely up-to-date
Ensure that only authorised users can send email through your system.
Limit outgoing email volumes
Accept reports of problems with your systems
Review the mail system logs on a regular basis
Be reliable (viz have at least 4 9s availability)
Don't be an open relay
Don't create backscatter
Maintain a good reputation
users can probably have acceptable success with all but the large email providers if they simply have their MTA hello as the name that the ISP assigns to the connection presuming that the ISP has forward and reverse DNS configured therefor.

Debian Apt Btrfs Auto-Snapshot Retrofit

nospam@example.com (Raymond P. Burkholder) — Mon, 10 Jul 2023 00:18:39 +0000

Debian with Automated Snapper Rollbacks is a short tutorial about setting up a Debian linux system with automated BTRFS snapshots of the system and easy rollback to previous auto-generated snapshots. Once it's setup, it'll automatically take pre/post snapshots when you run `apt` and you can boot them from grub.

Mail Verifier Tools

nospam@example.com (Raymond P. Burkholder) — Wed, 24 May 2023 00:24:17 +0000

email-security-scans.org
Port25’s Authentication and Spam Assassin Tool
verifier.port25.com - send an email to this address to check headers
multirbl.valli.org - "The complete IP check for sending Mailservers"
postmastery - Check the DNA of your email against important best practices using our free email deliverability test.
email-security-scans - Public Email & DNS Testbed - Considering just DKIM tests, this one is the only tester which explicitly and clearly recognizes ed25519 signatures. For the rest, EmailAudit reported a pass, but then said "DKIM key size is 0 bits. More senders are starting to use 2048 bits."
internet.nl - test website, email, connectivity
Wander Science DKIM Test
uriports - Test and optimize SPF, DMARC, DKIM, and MTA-STS
Google Admin ToolBox - Check MX - maybe be broken in some aspects, take with grain of salt
Easy DMARC - verify dns data
dmarcian - DMARC resources

Note: seems ';' (semi-colon) in records may help with some records.

send an e-mail to spf@tester.realsender.com
check online the validation results (it will take a minute to appear) realsender

Monitoring / Observability References

nospam@example.com (Raymond P. Burkholder) — Wed, 17 May 2023 23:50:04 +0000

I've been using check_mk for quite some time. It has progressed substantially recently and documentation has improved dramatically. However, their pythonesque configuration management has gone the way of the dinosaur, which has made my saltstack configuration management somewhat obsolete.

I'll have to recreate it as a REST interface solution somehow.

In the meantime, in looking at what else is out there, a few links:

OpenAPM - a site which shows the relationship between significant monitoring solutions - tools of interest: netdata, checkmk (doesn't really have many upstream/downstream ties), prometheus, rsyslog, collectd, jaeger, grafana
promethius
Jaeger - trace analysis
collectd - gathers metrics from various sources
grafana - feature rich metrics dashboard and graph editor for Graphite, Elasticsearch, OpenTSDB, Prometheus and InfluxDB

One of my current projects is for home automation. Currently it is based upon domoticz. Rather than relying on it's rather limited interface for data analysis, there are a couple of alternatives I'm thinking:

Using dashticz
cobbling together a series of best of class tools: TimescaleDB for holding timeseries data, telegraf which can take mqtt data streams and transmit to TimescaleDB, Grafana for handling querying and visualization of data. All the ZWave, Zigbee and other sensor data can be brought together for summary and for automation.
MQTT Consumer Telegraf Input Plugin
Monitoring temperature and humidity with MQTT - and using prometheus/grafana for back-end.

Prometheus appears to be popular for metric collection and distribution. Some add-ons for system monitoring for review:

MQTT2Prometheus - exporter translates from MQTT topics to prometheus metrics, written in Go. The exporter can only listen to one topic_path per instance.
Monitor IoT Devices in The Things Network with Prometheus and Grafana - extensive discussion on MQTT, Prometheus and Grafana with a little CBOR thrown in.
Since I use RabbitMQ to run my MQTT gateway, how appropo: RabbitMQ - Monitoring with Prometheus & Grafana
nut_exporter - Network UPS Tools (NUT) Prometheus Exporter [referenced from Prometheus Exporters and integrations]
Monitoring proxmox with prometheus and grafana - with a Proxmox specific dashboard

Random stuff:

sengled - light socket colour light bulbs for zigbee, bluetooth or wifi (zwave soon)
ZigBeeForDomoticz - Zigbee plugin for Domoticz. Allow to connect various zigbee controllers like Zigate but also Texas Instrument CC2531, CC13x2, CC26x2 ; Silicon-Labs; deConz based chipset to be connected to Domoticz [linked from Zigbee Device Compatibility Repository

Definition of Email List Washing

nospam@example.com (Raymond P. Burkholder) — Tue, 12 Apr 2022 03:59:18 +0000

From an email list, the definition of email list-washing:

Trying to identify the addresses in a spam list that will get you in trouble, e.g., spam traps and people with a history of complaining, so that you can mail to what's left. And also, by implication, dealing with complaints by saying "we took you off the list" without looking at the overall list quality.
The technical methods for removing stale bouncing addresses from a legitimate list are somewhat similar but the motivation is not.

Email Services

nospam@example.com (Raymond P. Burkholder) — Wed, 23 Feb 2022 17:14:50 +0000

Extracts from a mailop email list:

little known control panel, for creating such email/web servers: Aetolos - secure open source control panel for virtual hosting
Roundcube is the default webmail client these days, I am not sure if its a "replacement" for gmail, but it is very efficient and works great on mobile devices
Postfix+dovecot is the most common combination for running your own services, once combined they offer pretty much all the features most people need (SMTP/IMAP/POP3/etc)
ClamAV anti-virus is said to require at least 4GB of ram to load its default data set.
For a full featured modern package with reasonable spam resistance I can vouch for Mailu which requires Docker as a basis. Under the hood, this is postfix as MTA, dovecot for IMAP/POP3, rspamd as anti-spam solution (I think the SPF/DKIM/DMARC stuff is located there, too), Roundcube or RainLoop as webmail (both pretty usable, probably not as feature-rich as GMail), PostgreSQL for account persistence, and REDIS as memory cache mostly for the rspamd engine. Supports Letsencrypt out of the box, of course. It does take some effort to set up the basics right (Docker and configuration) but then runs very reliably and is a breeze to manage. Sadly, it does not integrate Mailman, so I had to do that manually, which is kind of a pain.
If you have a reasonable insight into unix things, you could do worse than going with OpenBSD and an OpenSMTPD setup along the lines of what Aaron Poffenberger describes in his tutorial here: opensmtpd for the real world mail server tutorial (originally a BSDCan session I believe).
I’ve been running similar setups for years (but with exim as the MTA mainly for inertia reasons), documented mainly in articles you can find via the first link in my signature. The most comprehensive is perhaps Effective Spam and Malware Countermeasures - Network Noise Reduction Using Free Tools , others will be tagged with keywords like spam, mail, spamd, smtp and so forth.
Gmail's webmail interface is one of the worst possible. It is very inefficient to operate, counter-intuitive, hides many important information from the user etc., not mentioning that it is simply ugly. There are many much nicer and more user-friendly webmails, like for example already mentioned Roundcube.
Here I found an overview of various webmail apps, with screenshots of some. You can pick something that suits you: Top 15 Self-hosted open-source free web-based email clients
I've begun with OpenSMTP, but I've switched to Postfix as the primary developer has stopped actively working on it. I now run Postfix, Dovecot, Solr (for the search), and Rspamd, and it's working perfectly!
By the time you go through all the hassle of managing spam filters and getting your IP warmed up, Fastmail at $50/yr/mailbox looks pretty attractive.
If you can find someone who resells Tucows' white label e-mail, they have a pretty good product for about $10/mailbox/yr for 5GB, $20 for 10GB, $30 for 15GB.
take a look at mailcow dockerized. It's all you need (mail server, sogo groupware, antispam/-virus, 2fa,...). Put it on a VPS or your NAS and be happy.

Network UPS Tools - Diagnostics

nospam@example.com (Raymond P. Burkholder) — Thu, 30 Dec 2021 21:47:35 +0000

Some diagnostic tools to remember:

$ sudo upsc apc@localhost

battery.charge: 100
battery.charge.low: 10
battery.charge.warning: 50
battery.mfr.date: 2014/07/01
battery.runtime: 1260
battery.runtime.low: 120
battery.temperature: 22.5
battery.type: PbAc
battery.voltage: 27.7
battery.voltage.nominal: 24.0
....
ups.beeper.status: enabled
ups.delay.shutdown: 20
ups.delay.start: 30
ups.firmware: 653.19.D
ups.firmware.aux: 11.03
ups.load: 13.0
ups.mfr: American Power Conversion
ups.mfr.date: 2011/11/03
ups.model: Smart-UPS 1500
ups.productid: 0002
ups.serial: AS1145221386
ups.status: OL
ups.test.result: No test initiated
ups.timer.reboot: -1
ups.timer.shutdown: -1
ups.timer.start: -1
ups.vendorid: 051d

$ sudo upscmd -l apc@localhost

Instant commands supported on UPS [apc]:

beeper.disable - Disable the UPS beeper
beeper.enable - Enable the UPS beeper
beeper.mute - Temporarily mute the UPS beeper
beeper.off - Obsolete (use beeper.disable or beeper.mute)
beeper.on - Obsolete (use beeper.enable)
load.off - Turn off the load immediately
load.off.delay - Turn off the load with a delay (seconds)
load.on - Turn on the load immediately
load.on.delay - Turn on the load with a delay (seconds)
shutdown.reboot - Shut down the load briefly while rebooting the UPS
shutdown.return - Turn off the load and return when power is back
shutdown.stayoff - Turn off the load and remain off
shutdown.stop - Stop a shutdown in progress
test.battery.start.deep - Start a deep battery test
test.battery.start.quick - Start a quick battery test
test.battery.stop - Stop the battery test
test.panel.start - Start testing the UPS panel
test.panel.stop - Stop a UPS panel test

A list of possible variables: C.1. Variables

To find out what variables are in use by which UPS, visit Devices Dumps Library

UPS monitoring with NUT, Nagios and Check_MK

Spam Fighting Resources

nospam@example.com (Raymond P. Burkholder) — Sat, 16 Oct 2021 18:04:22 +0000

Some recent suggestions I've encountered:

SpamAssassin KAM Rules - Just a note that the much better way to use the rules is with the KAM Channel, see KAM Channel
Sane Security Signatures

Rspamd is not a dead project and it can do all the stuff OpenDKIM+OpenDMARC+OpenARC can do (yes, even ARC signing and DMARC reporting). I have even added support of OpenDKIM style signing tables [1].
So far, Rspamd supports ed25519 and dual-signing out of the box. It also supports Hashicorp Vault to store (and rotate!) DKIM secrets [2]. In terms of DKIM signing there is also support of fast signing (with all content scan features disabled), so it is highly unlikely that it's performance can be beaten by any other signing tool in this mode [3].

Use of Signing Table
DKIM signing using Vault
Optimizing Signing Only Mode

Rspamd (embedded in a Mailu installation) for low-maintenance operation. That packages includes several mechanisms to score messages and handle them according to score intervals. This does a relatively good job but isn't really able to detect persistent spammers using their own IP ranges.

Are you aware of its reputation module? In conjunction with selector system you can build reputation on wide range of parameters without touching Lua. Beside its IP reputation i use sender's eSLD, IPnet & ASN reputation with score limited to not be enough for rejection.
If spammer uses not related IP, ASN and/or senders (snowshoe), you still can use its bayes and/or fuzzy checks. But basically the reputation module with RBL scoring does good job and in last year i do not remember any false negative mail and small amount of false positives, especially from gmail, these false positives are mostly not due reputation, but due SPF/DKIM/DMARC ignorance of our eshop systems.
I have small python script, which gets reputation data from rspam's redis DB and fill dynamic blocklist for exim, to block persistent spammers at MTA level (RCPT stage).

Anyway, I document what I do in a blog post: Zimbra AntiSpam Best Practices
The post describes how we rely on several layers of protection (including those provided by Postfix, which I know you are not using), as well as our adjustments to Amavis and SpamAssassin. I hope there are some techniques in the blog post that will be helpful to you.

For Let's Encrypt certificates, I use these awesome scripts, they are written as bash shell scripts and they are infinitely better than the official certbot tool, they can be used without a web server, by using DNS API integration. They are highly recommended:

acme.sh
dehydrated

For the first one, it no longer defaults to LE, it uses zerossl, because its rights are now owned by them. You can re-enable LE as default though by:
acme.sh --set-default-ca --server letsencrypt

I use Bruce Guenter's mailfront for my incoming mail so I had to add SNI support. It wasn't very hard. [when using SNI to differentiate between certificates]

if one abuses any of 25|465|587 they are blocked on all of 25/465/587 with a lengthy filter time
if one abuses any of 110|143|993|995 they are blocked on all of 110/143/993/995 with a slightly shorter filter time

Jaeger - Distributed Tracing

nospam@example.com (Raymond P. Burkholder) — Sun, 12 Sep 2021 16:56:00 +0000

Some articles I've been collecting for observability / distributed tracing:

How to maximize span ingestion while limiting writes per second to Scylla with Jaeger - Jaeger primarily supports two backends: Cassandra and Elasticsearch. Here at Grafana Labs we use Scylla, an open source Cassandra-compatible backend. In this post we’ll look at how we run Scylla at scale and share some techniques to reduce load while ingesting even more spans. We’ll also share some internal metrics about Jaeger load and Scylla backend performance.
The Cost of Containerization for Your Scylla - There is, however, a performance payoff for the operational convenience of using containers. This is to be expected because of the extra layer of abstraction (the container itself), relaxation on resource isolation, and increased context switches.
Hacking your way to Observability — Part 3 - A distributed tracing quick start with Jaeger and OpenTelemetry

2021/09/26 - Two different links from two different lists talking about the same thing but differently:

Five years evolution of open-source distributed tracing - provides temporal context for Hawkular, Zipkin, OpenTracing, Jaeger, OpenTelemetry, Hypertrace, Grafana Tempo and SigNoz
Observability: The 5-Year Retrospective - how to find answers to unknown-unknowns.

2021/10/28 Hacker News - OpenTelemetry

OpenTelemetry was OpenCensus, and before that it was OpenTracing
OpenTelemetry is a collection of tools, APIs, and SDKs. Use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) to help you analyze your software’s performance and behavior.
OpenMetrics relates to the metrics part of OpenTelemetry, where the latter is currently in alpha/experimental phase. OpenTelemetry has a statement on being interoperable with OpenMetrics for both accepting, forwarding and generating in the OpenMetrics format [0]. So they are two separate CNCF projects where OpenMetrics has overlap with a part of OpenTelemetry.

Software Defined Radio

nospam@example.com (Raymond P. Burkholder) — Sat, 20 Mar 2021 21:00:35 +0000

phoronix had an item on Software-Defined Radio Benchmarks, Other Updates This Week in terms of, since SDR is becoming popular and compute intensive, phoronix has added bencharks to their workload test tools.

Some interesting links originating from here:

Open Benchmarking - LuaRadio - is a lightweight software-defined radio (SDR) framework built atop LuaJIT. LuaRadio provides a suite of source, sink, and processing blocks, with a simple API for defining flow graphs, running flow graphs, creating blocks, and creating data types.
Open Benchmarking - gnuradio - is a free & open-source software development toolkit that provides signal processing blocks to implement software radios.
Open Benchmarking - srsLTE - Open-source LTE software radio suite developed by Software Radio Systems (SRS).
Open Benchmarking - liquidsdr - is home to liquid-dsp, a free and open-source signal processing library for software-defined radios written in C. Its purpose is to provide a set of extensible DSP modules that do not rely on external dependencies or cumbersome frameworks.

Somewhat associated: Noise Mapping with KSQL, a Raspberry Pi and a Software-Defined Radio - ADS-B capture, RTL2832U radio

ZRAM

nospam@example.com (Raymond P. Burkholder) — Fri, 10 Apr 2020 15:29:13 +0000

zram, formerly called compcache, is a Linux kernel module for creating a compressed block device in RAM, i.e. a RAM disk, but with on-the-fly “disk” compression. Debian has a init system file for it.

I'm one of those who use zsmalloc as a module - mainly because I use zram as a compressing general purpose block device, not as a swap device. I create zram0, mkfs, mount, checkout and compile code, once done - umount, rmmod. This reduces the number of writes to SSD. Some people use tmpfs, but zram device(-s) can be much larger in size. That's a niche use case and I'm not against the patch.

Running out of RAM in Ubuntu? Enable ZRAM -

Dell R610 with H310 Drive Controller

nospam@example.com (Raymond P. Burkholder) — Thu, 26 Mar 2020 18:01:33 +0000

When using some relatively inexpensive parts to build a test three server Ceph solution, it seems that drive controllers and SSD drive combinations are finicky. Specifically, it seems that a Dell H310 controller may have issues with some or all Samsung SSD drives.

At Slow writes PERC H700 and Samsung 850 PRO SSD, a Dell response indicates:

The Samsung 850 Pro SSDs are not validated or certified to work with Dell controllers and as such there is a communication mismatch between the drives and the controller at the firmware level. As a result, you are bound to realize unexpected poor Read and Write performance regardless of controller cache settings.

It said that the Samsung 840 drive will work with the controllers. With Broadcom taking over LSI, they've removed old links, so documents are hard to find now. In trying to follow a document trail:

Dell PowerEdge RAID Controller (PERC) H310, H710, H710P, and H810User’s Guide on page 9 says the H310 has an LSI2008 chipset. I'm looking at the H310 as it provides for direct passthrough to the drives with out RAID. The card has no caching.
LSI SAS 2008 RAID Controller/ HBA Information - An H310 is similar to the LSI9211-8i
Check Interoperability and Compatibility

To use the H310 for non-cache passthrough, it is recommended to flash the card in IT mode. I'll try this once I obtain the card. In the meantime, some possibly relevant links to the drivers and process:

Disk Controller features and Queue Depth? - from 2014, discusses queue depth and diagnostic commands.
Flashing the H310 Mono Mini to IT mode has a link to probably the best link to a flashing tutorial. But is complicated for flashing the mini H310. From the notes: the R610 is an 11th generation server, and when the mini H310 is reflashed to IT mode, the server probably won't reboot.
Crossflashing the Dell PERC H200 and H310 to the LSI 9211-8i (from year 2018) implies that the H310 is the equivalent of an LSI 9211-8i. Also, a Dell H200 and an H310 must be similar from a firmware perspective.
Flashing IT Firmware to the LSI SAS9211-8i HBA - 2012 - flash to a pci card
How to crossflash PERC H310 to IT mode LSI 9211-8i firmware (HBA for FreeNAS, UnRAID) - an article from 2017 with better, more native instructions
Crossflash Dell PERC H310 to LSI 9211-8i IT Mode Using Legacy (DOS) and UEFI Method (HBA Firmware + BIOS)
PERC H310 - LSI 9211-8i - $50 - notes on purchase, install, and upgrade of card.
Problems Flashing Dell Perc H310 - discusses H310 in PCI slot instead
How-to Flash Dell Perc H310 with IT Firmware To Change Queue Depth from 25 to 600
Updated: SAS HBA crossflashing or flashing to IT mode, Dell Perc H200 and H310 from 2016 where is is suggested that "Integrated, Mini or Mini Mono Perc H310 do NOT try to crossflash with these steps".
DELL PERC H310/H710/H710P/H810 Controllers driver version 5.1.112.64,A00 - a starting point for the search of appropriate drivers
LINUX PERCCLI Utility For All Dell HBA/PERC Controllers - and the starting point for command line utilities
LSI MegaRAID SAS - some notes on the CLI utilities
Doing battle with a Dell R620 and Ubuntu - has some remarks on H310 testing.
H310 and Fan Noise
Low random read/write on samsung 840 250gb ssd - spectre bios upgrades can destroy random read/write times

Drive Information:

Cluster test:

rados bench -p vm_storage 10 write -b 4M -t 16"

2022/04/13 - challenges

In a Dell R620, with an LSI SAS card, it took a while to find a utility to communicate with the card. Over the fold are some results:
Continue reading "Dell R610 with H310 Drive Controller"