Raymond P. Burkholder - Things I Do

unstruct.ai - An AI-Enabled, Open-Source Alternative to PagerDuty. With UnStruct.AI, you're not just getting another cybersecurity tool – you're getting an all-in-one powerhouse. Instead of juggling multiple tools and racking up costs for each, get everything under one roof. Whether it's for paging, incident response, analysis, status updates, SLO/uptime monitoring, or a sprinkle of tech magic.

Hacker News Perspective

According to the slides from a 2023 Linux Storage, Filesystem, Memory-Management and BPF Summit talk, guests operating through the netkit device (which was called "meta" at that time) are able to attain TCP data-transmission rates that are just as high as can be had by running directly on the host. The performance penalty for running within a guest has, in other words, been entirely removed.

Chamberlain blocks smart garage door opener from working with smart homes - Chamberlain Group recently made the decision to prevent unauthorized usage of our myQ ecosystem through third-party apps.

Whenever someone says: I can control my garage door from my phone! I point out that what they've really done is:

• ceded all control of their garage door to another entity
• they request action(s) from that entity
• hope the other entity allows that action

One popular fix for people with the MyQ problem is ratgo - You plug a board into the opener and it talks to you locally with no cloud involvement.

• Learn Nokia SR Linux - Service Router Linux
• SR tutorial with containerlab
• Microsoft has crafted a switch OS on Debian Linux

• a href="https://www.first.org/resources/papers/aa-dec2021/Protective-DNS-a-Boris-Slides.pdf" target=_blank>Protective DNS – Why It Matters, How to Deploy It On-prem, and to Take Control and Defense back
• Threat blocking
• Introducing 1.1.1.1 for Families
• Cybercrime Supply Chain 2023: Measurements and Assessments of Cyber Attack Resources and Where Criminals Acquire Them

At one point Firefox was going to send everything to their favorite DoH resolver but they got a great deal of pushback from people who pointed out that they had policies on their networks and they'd have to ban Firefox. Firefox responded with a lame hack where you can tell your cache to respond to some name and if so Firefox will use your resolver.

To install rabbitmq and enable mqtt extensions:

• apt install rabbitmq-server
• rabbitmq-plugins enable rabbitmq_mqtt

To add a user and set permissions for an mqtt client, only one client can use the connection at once

# username and password are both "mqtt-test"
rabbitmqctl add_user mqtt-test mqtt-test
rabbitmqctl set_permissions -p / mqtt-test ".*" ".*" ".*"
rabbitmqctl set_user_tags mqtt-test management   

To disable anonymous attachments:

root@rabbitmq:/# cat /etc/rabbitmq/rabbitmq.conf
mqtt.allow_anonymous = false

how to do a dynamic parser in X3?

The presentation went well, but as always with a large surface area library like Spirit it went on for too long IMO (2 hrs). This was similar to what happened when I presented Spirit 2.x almost 10 yrs ago. If I were to present on this again, I would split the topic into two talks:

• A background on recursive variant datatypes and visitor pattern and dipping into the Fusion library for adapting arbitrary application structures to Fusion tuples.
• Spirit X3 parsing w/annotations and error handling.

I try to present from real code in the IDE as much as possible so that we're looking at real code. I almost always write my own examples with the library I'm presenting and put them on github. The exception is when the library comes with an example that is everything I want to talk about .

I cribbed quite a bit from the spirit x3 fun example presented by Michael Caisse a few years back, but I did extend the AST to handle imaginary numbers.

How to use SkyScanner to find Cheap Flights

Request: Or people simple do not want to share own email, as it is not related for delivery, but required by the form. Thus they fill semi random email like string.

Response: That is what mailinator is for, or the temporary addresses offered by several browser plugins, or if you know they're going to use it to spam you, abuse@ their domain. nobody@example.com works as well, and 1-900-976-1212 for a "required" phone number.

Alioth: A Machine Learning Based Interference-Aware Performance Monitor for Multi-Tenancy Applications in Public Cloud

Multi-tenancy in public clouds may lead to co-location interference on shared resources, which possibly results in performance degradation of cloud applications. Cloud providers want to know when such events happen and how serious the degradation is, to perform interference-aware migrations and alleviate the problem. However, virtual machines (VM) in Infrastructure-as-a-Service public clouds are black-boxes to providers, where application-level performance information cannot be acquired. This makes performance monitoring intensely challenging as cloud providers can only rely on low-level metrics such as CPU usage and hardware counters.

We propose a novel machine learning framework, Alioth, to monitor the performance degradation of cloud applications. To feed the data-hungry models, we first elaborate interference generators and conduct comprehensive co-location experiments on a testbed to build Alioth-dataset which reflects the complexity and dynamicity in real-world scenarios. Then we construct Alioth by (1) augmenting features via recovering low-level metrics under no interference using denoising auto-encoders, (2) devising a transfer learning model based on domain adaptation neural network to make models generalize on test cases unseen in offline training, and (3) developing a SHAP explainer to automate feature selection and enhance model interpretability. Experiments show that Alioth achieves an average mean absolute error of 5.29% offline and 10.8% when testing on applications unseen in the training stage, outperforming the baseline methods. Alioth is also robust in signaling quality-of-service violation under dynamicity. Finally, we demonstrate a possible application of Alioth's interpretability, providing insights to benefit the decision-making of cloud operators. The dataset and code of Alioth have been released on GitHub.

There was a note on reddit/r/debian which states that Wireguard is fully integrated into the Linux Kernel as of kernel v5.10. I suppose I could have saved a bunch of drama with upgrading to Bookworm which has kernel v6.1 natively, by instead using Bullseye-Backports, but I decided to go all the way. Hindsight is 20/20. A few other machines were already running Bookworm so I thought I had no problems.

It is nice to see that wireguard-tools references nftables. And there are a number of examples as reference for various scenarios.

So, with Wireguard in the kernel, no dkms installation is required. Just the installation of the tools (assumes root or sudo). Use the --no-install-recommends, otherwise your kernel will be replaced with a real-time kernel.

# apt install --not-install-recommends wireguard-tools
# cd /etc/wireguard

Create the keys for a peer to peer session:

# wg genkey | tee key_server_private | wg pubkey > key_server_public
# wg genkey | tee key_client_private | wg pubkey > key_client_public
# chmod -v 600 key*
# ls -al /etc/wireguard/
  total 20
  drwx------ 1 root root   54 Jul 18 04:30 .
  drwxr-xr-x 1 root root 2348 Jul 19 01:29 ..
  -rw------- 1 root root   45 Jul 19 02:49 key_client_private
  -rw------- 1 root root   45 Jul 19 02:49 key_client_public
  -rw------- 1 root root   45 Jul 19 02:49 key_server_private
  -rw------- 1 root root   45 Jul 19 02:49 key_server_public

A sample edge interface for server side termination of VPN (file name: wg0.conf):

[Interface]
Address = 10.20.10.1/24
#SaveConfig = true
ListenPort = 51820
PrivateKey = <server private key>

[Peer]
PublicKey = <client public key>
AllowedIPs = 10.20.10.0/24

A sample client interface, say on an Android for connection back to the server side (flie name: wg0-client.conf):

[Interface]
Address = 10.20.10.11/24
PrivateKey = <client private key>
DNS = 10.10.30.100

[Peer]
PublicKey = <server public key>
Endpoint = <server outside address>:51820
AllowedIPs = 10.20.10.0/24, 10.10.0.0/16
PersistentKeepalive = 21

If the allowed address is '0.0.0.0/0', then all traffic goes through the VPN. Use ipv6-test.com or ipleak.net to verify that traffic is going trough the VPN, or use something like WhatIsMyIpAddress.

Impressively, someone has created a QR generator which will generate a code to the terminal window (not a graphic file, but an ansii thingy in a terminal window). This can then be scanned by Android WireGuard to load the configuration.

$ qrencode -t ansiutf8 < wg-android.conf

I use a saltstack script to build a zone based firewall composed of nftable rules. Basically two rules are needed: a) burn a port through the firewall, and b) allow access to the interior network sections for one or all ports.

To turn on the interface and start it automatically:

# chmod -v 600 /etc/wireguard/wg0.conf
# wg-quick up wg0
# systemctl enable wg-quick@wg0.service

To turn off the interface and keep it off:

# wg-quick down wg0
# systemctl disable wg-quick@wg0.service

To show connections and status:

# wg show
interface: wg0
  public key: <server public key>
  private key: (hidden)
  listening port: 51820

peer: 
  endpoint: :4496
  allowed ips: 10.20.10.0/24
  latest handshake: 44 minutes, 26 seconds ago
  transfer: 2.50 MiB received, 33.47 MiB sent

With the SaveConfig enabled, more clients can be added and saved:

# wg genkey | tee key_mac_private | wg pubkey > key_mac_public 
# wg set wg0 peer <mac public key> allowed-ips 10.20.10.12/32

Stan's Blog mentioned terminating the server side VPN on UDP port 53. Many/Most networks allow this out, so would/could be a way out of a heavily protected network to the destination.

Note: this config was added to a privileged lxc container, nothing special was required for building the wireguard interface.

I found out the hard way that SaltStack and Debian no longer place nice together. I had upgraded a Debian installation from Bullseye to Bookworm, along with the resident Salt Minion. When attempting to use the minion, it no longer starts up, due to various imports no longer working. Which was due to the salt-minion not being upgraded. The error message would started this odyssey:

salt ImportError: cannot import name 'Markup' from 'jinja2'

Taking a look at the Debian Developer Information for Salt, the last version started in 'unstable' was 3004.1 back in December of 2022. This is now almost 8 months later and little or no movement. There was some mention in a ticket somewhere that Salt release cycles don't cater to Debian stable release cycles. Not sure if that is a legitimate reason or not, but, well, for whatever reason, SaltStack management in Debian is no longer a simple no brainer.

However, after a little digging, there is a way to run SaltStack versions 3006 (current as of this writing). It is simple to install on Bullseye, but not easily done on Bookworm.

On Bullseye (as root, or implies sudo):

# cd ~ # apt remove salt-minion salt-master # apt install curl # curl -L https://bootstrap.saltstack.com -o install_salt.sh # sh install_salt.sh -M onedir

The '-M' installs the salt master at the same time (for machines running master). If you forget to do that, you'll need to diagnose and fix the systemctl mask error with the following:

# apt install file # file /etc/systemd/system/salt-master.service # rm /etc/systemd/system/salt-master.service # systemctl daemon-reload # sh install_salt.sh -M onedir

The 'sh install_salt.sh -M onedir' should show a symlink to /dev/nul, which the 'rm ...' will fix.

On Bookworm, the bootstrap isn't scheduled to work till beginning of 2024 sometime I think with Salt 3007 or 3008 -- more info in [FEATURE REQUEST] Add Salt support for Debian 12 #64223 .

In the meantime, I had to cheat a bit:

• in /etc/debian_version, change 12.0 to 11.0
• in /etc/apt/sources.list, change bookworm to bullseye
• rm /etc/apt/sources.list.d/salt.list
• run apt update
• run the commands listed above for installing the one or both the salt services
• restore /etc/debian_version and /etc/apt/sources.list to their original content

I'm sure there are more elegant ways of doing this, but this worked to fake the needed version 11 in the installation script and directory traversal requirements

Note, more info on the Salt Install/Bootstrap Process.

Market Making with Deep Reinforcement Learning from Limit Order Books

Market making (MM) is an important research topic in quantitative finance, the agent needs to continuously optimize ask and bid quotes to provide liquidity and make profits. The limit order book (LOB) contains information on all active limit orders, which is an essential basis for decision-making. The modeling of evolving, high-dimensional and low signal-to-noise ratio LOB data is a critical challenge. Traditional MM strategy relied on strong assumptions such as price process, order arrival process, etc. Previous reinforcement learning (RL) works handcrafted market features, which is insufficient to represent the market. This paper proposes a RL agent for market making with LOB data. We leverage a neural network with convolutional filters and attention mechanism (Attn-LOB) for feature extraction from LOB. We design a new continuous action space and a hybrid reward function for the MM task. Finally, we conduct comprehensive experiments on latency and interpretability, showing that our agent has good applicability.

• make sure the MTA's forward and reverse DNS match
• set up an SPF record, probably "v=spf1 mx ~all"
• manage DNS MX, DKIM, DMARC and SPF records (SPF, DKIM, and DMARC are a recommended order of implementation)
• set up DKIM signing for each domain you host, make the DKIM domain match the From: domain
• start slow and look at any bounces
• maybe collect DMARC stats but for a small volume MTA, not very interesting
• instead of struggling with Postfix, OpenDKIM, Dovecot and friends (and losing out on quite a few features), try looking at maddy
• arrange for a backup MTA
• manage reverse lookup records, including managing the uncertain chain of authority between the instance and the nearest SOA
• manage certificates associated with TLS for SMTP and IMAP
• manage DKIM certificate
• manage one's upstream to address PBL issues
• keep the MTA secure and free from DOS attack
• Use a static IPv4 address for your email system [counter point: If DNS is fully functional and IP addresses don't change too quickly and TTL is configured properly on DNS records, ... then why is a static IP address strictly required? ]
• Do not share this IPv4 address with user machines
• Do not host your email system 'in the cloud'
• Make sure that your IP address is not listed in the PBL
• Provide an MX record
• Provide meaningful and consistent reverse DNS
• Your system should say HELO (or EHLO) with its hostname
• Keep your software completely up-to-date
• Ensure that only authorised users can send email through your system.
• Limit outgoing email volumes
• Accept reports of problems with your systems
• Review the mail system logs on a regular basis
• Be reliable (viz have at least 4 9s availability)
• Don't be an open relay
• Don't create backscatter
• Maintain a good reputation
• users can probably have acceptable success with all but the large email providers if they simply have their MTA hello as the name that the ISP assigns to the connection presuming that the ISP has forward and reverse DNS configured therefor.

Debian with Automated Snapper Rollbacks is a short tutorial about setting up a Debian linux system with automated BTRFS snapshots of the system and easy rollback to previous auto-generated snapshots. Once it's setup, it'll automatically take pre/post snapshots when you run `apt` and you can boot them from grub.