Entries by Raymond P. Burkholder

Learn something, a parameter for dmesg to look for certain types of messages:

root@host02:~# dmesg --level err
[    0.239659] ACPI: SPCR: Unexpected SPCR Access Width.  Defaulting to byte size
[    2.676888] mpt2sas_cm0: overriding NVDATA EEDPTagMode setting
[   11.513804] ACPI Error: No handler for Region [SYSI] (00000000680cfabd) [IPMI] (20210730/evregion-130)
[   11.513921] ACPI Error: Region IPMI (ID=7) has no handler (20210730/exfldio-261)
[   11.514031] ACPI Error: Aborting method \_SB.PMI0._GHL due to previous error (AE_NOT_EXIST) (20210730/psparse-529)
[   11.514168] ACPI Error: Aborting method \_SB.PMI0._PMC due to previous error (AE_NOT_EXIST) (20210730/psparse-529)

Man page has various level categories:

• subsys - The message sub-system prefix (e.g., "ACPI:").
• time - The message timestamp.
• timebreak - The message timestamp in short ctime format in --reltime or --human output.
• alert - The text of the message with the alert log priority.
• crit - The text of the message with the critical log priority.
• err - The text of the message with the error log priority.
• warn - The text of the message with the warning log priority.
• segfault - The text of the message that inform about segmentation fault.

Hacker News had an entry today on Text Only News WebSites - a fast way to get to the meat of the matter.

I wish it was RSS like, but there are alternatives for summaries (independent of anything yahoo/google/facebook might produce):

• The Brutalist Report - snapshot of of what is happening across various news sources
• skimfeed - snapshot of what is happening across various news sources
• github: awesome console services - meta link to find more console stuff
• Another take on from the seeing impaired, something we can learn: "I’ve been getting a lot mileage out of safari’s and brave’s reader mode"
• markets.sh - meh, summaries might be ok, but can't copy text for stuff I want to keep
• teddit worldnews - kinda like a reddit thingy, but not sure how it works

Now if I could find something that scrolled as stuff changes, .... oh wait, the closest would be irc.libera.chat with ##hntop (hackernews), ##news (world news), ##alerts (earthquakes, electrical, ...)

Obtaining interface list, which can be used to drill down into details:

$ ls /sys/class/net/
br0  eno1  enp5s0  lo  lxcbr0  veth-tf64-v90  vlan90  vlan90_br0  wlp6s0
rpb@nuc:~/data/passwords$ ls /sys/class/net/enp5s0
addr_assign_type  carrier             dev_id    gro_flush_timeout  master                operstate       proto_down  testing       upper_br0
address           carrier_changes     dev_port  ifalias            mtu                   phys_port_id    queues      threaded
addr_len          carrier_down_count  dormant   ifindex            name_assign_type      phys_port_name  speed       tx_queue_len
broadcast         carrier_up_count    duplex    iflink             napi_defer_hard_irqs  phys_switch_id  statistics  type
brport            device              flags     link_mode          netdev_group          power           subsystem   uevent

Provides some descriptive details and places of interest:

$ sudo udevadm test-builtin net_id /sys/class/net/enp5s0
Trying to open "/etc/systemd/hwdb/hwdb.bin"...
Trying to open "/etc/udev/hwdb.bin"...
Trying to open "/usr/lib/systemd/hwdb/hwdb.bin"...
Trying to open "/lib/systemd/hwdb/hwdb.bin"...
Trying to open "/lib/udev/hwdb.bin"...
=== trie on-disk ===
tool version:          252
file size:        12198286 bytes
header size             80 bytes
strings            2478998 bytes
nodes              9719208 bytes
Loading kernel module index.
Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy
Found container virtualization none.
Using default interface naming scheme 'v252'.
Parsed configuration file "/usr/lib/systemd/network/99-default.link"
Parsed configuration file "/usr/lib/systemd/network/73-usb-net-by-mac.link"
Created link configuration context.
ID_NET_NAMING_SCHEME=v252
ID_NET_NAME_MAC=enx54b2030473fa
enp5s0: MAC address identifier: hw_addr=54:b2:03:04:73:fa → x54b2030473fa
ID_OUI_FROM_DATABASE=PEGATRON CORPORATION
sd-device: Failed to chase symlinks in "/sys/devices/pci0000:00/0000:00:1c.1/0000:05:00.0/of_node".
sd-device: Failed to chase symlinks in "/sys/devices/pci0000:00/0000:00:1c.1/0000:05:00.0/physfn".
enp5s0: Parsing slot information from PCI device sysname "0000:05:00.0": success
enp5s0: dev_port=0
enp5s0: PCI path identifier: domain=0 bus=5 slot=0 func=0 phys_port= dev_port=0 → p5s0
ID_NET_NAME_PATH=enp5s0
Unload kernel module index.
Unloaded link configuration context.

The raw details contributing to the previous data:

$ udevadm info /sys/class/net/enp5s0
P: /devices/pci0000:00/0000:00:1c.1/0000:05:00.0/net/enp5s0
M: enp5s0
R: 0
U: net
I: 2
E: DEVPATH=/devices/pci0000:00/0000:00:1c.1/0000:05:00.0/net/enp5s0
E: SUBSYSTEM=net
E: INTERFACE=enp5s0
E: IFINDEX=2
E: USEC_INITIALIZED=5978173
E: ID_NET_NAMING_SCHEME=v252
E: ID_NET_NAME_MAC=enx54b2030473fa
E: ID_OUI_FROM_DATABASE=PEGATRON CORPORATION
E: ID_NET_NAME_PATH=enp5s0
E: ID_BUS=pci
E: ID_VENDOR_ID=0x8086
E: ID_MODEL_ID=0x157b
E: ID_PCI_CLASS_FROM_DATABASE=Network controller
E: ID_PCI_SUBCLASS_FROM_DATABASE=Ethernet controller
E: ID_VENDOR_FROM_DATABASE=Intel Corporation
E: ID_MODEL_FROM_DATABASE=I210 Gigabit Network Connection
E: ID_PATH=pci-0000:05:00.0
E: ID_PATH_TAG=pci-0000_05_00_0
E: ID_NET_DRIVER=igb
E: ID_NET_LINK_FILE=/usr/lib/systemd/network/99-default.link
E: ID_NET_NAME=enp5s0
E: SYSTEMD_ALIAS=/sys/subsystem/net/devices/enp5s0
E: TAGS=:systemd:
E: CURRENT_TAGS=:systemd:

On Debian Linux, fonts can be downloaded and installed in the user's directory, and do not need to be installed into the protected system directories.

Ensure the local directory is available:

mkdir -p ~/.local/share/fonts

Font files can be unzipped and the .ttf files can be copied over into that directory

Refresh the font cache

fc-cache -f -v

Review the fonts:

fc-list

A number of fonts can be installed with:

apt install ttf-mscorefonts-installer fonts-aenigma

1001 Fonts has a wide range of fonts compatible with Linux.

As a side note, pixabay has a good variety of clipart selections.

Options orders aren't hidden, but bid/ask quotes are there only because market makers need to post them, and they do try to get the best price for themselves while not getting screwed in case of news/events.

But they continually watch the incoming orders and compete for them. When you submit an order, there is a quick 100-millisecond auction between MMs and if your price is good enough then one of them wins, and may even give you better price than you asked. Otherwise your order becomes the best bid or ask/offer and even skews the mid. At that moment the previous "mid" doesn't matter because your order affects it and you've created a new "mid". At some point someone may simply match your price, or a market maker may fill it when they find another order or a hedge that still makes them a few bucks. Basically your order needs to be useful to someone else to get filled, and this has nothing to do with recent bid & ask. Most orders do get filled near the mid, the only question is how far from the mid.

Found at Elite Trader - BID/ASK spread on SPX vs ES options

Light Polution Map

Supplement to: The New World Atlas of Artificial Night Sky Brightness - where they get their data set

Bortle Scale - a way of measuring the quality (brightness) of the night sky for a particular location. There are nine levels to the Bortle scale with Class 9 being the most extreme amount of light pollution.

Load balancing is prevalent in practical application (e.g., web) deployments seen today. One such load balancer, HAProxy, remains relevant as an open-source, easy-to-use system. In the context of web systems, the load balancer tier possesses significant influence over system performance and the incurred cost, which is decisive for cloud-based deployments. Therefore, it is imperative to properly tune the load balancer configuration and get the most performance out of the existing resources. In this technical report, we first introduce the HAProxy architecture and its load balancing methods. Then, we discuss fine-tuning parameters within this load balancer and examine their performances in face of various workload intensities. Our evaluation encompasses various types of web requests and homogeneous and heterogeneous back-ends. Lastly, based on the findings of this study, we present a set of best practices to optimally configure HAProxy.

Load Balancer Tuning: Comparative Analysis of HAProxy Load Balancing Methods

• What is Kubernetes and how should you monitor it? - A monitoring strategy that takes advantage of Kubernetes will give you a bird's eye view of your entire application’s performance, even if containers running your applications are continuously moving between hosts or being scaled up and down.
• How to optimize Kubernetes resource configurations for cost and performance - in part two of this Kubernetes guide, you'll get help balancing appropriate parameter configuration for any cluster you are working with now or in the future. You'll learn about requests and limits, measuring CPU utilization, and how to optimize Kubernetes resource allocation.
• Best way to install and use kubernetes for learning - reddit
• Kubernetes The Hard Way - This tutorial walks you through setting up Kubernetes the hard way. This guide is not for people looking for a fully automated command to bring up a Kubernetes cluster. Kubernetes The Hard Way is optimized for learning, which means taking the long route to ensure you understand each task required to bootstrap a Kubernetes cluster.
• Deploying a Kubernetes Cluster within Proxmox using Ansible

• A capable attacker will look for plaintext
• red/black separation
• Schneiers's principle
• Zooko's tradeoff
• Kerckhoffs's Principle
• Corollary to Metcalfe's principle [2]
• It is easier for insiders to steal information - also janitors, cleaners
• Design for known threats
• Design for future threats
• Design for unknown threats as far as possible
• existing systems persist
• defence in depth
• monoculture -> target more attractive, usually more brittle
• the capital and operating costs of well-designed secure systems are about the same as those of insecure ones until the insecure ones fail
• keep intrusion records
• keep i/o records
• cheap and effective security needs good system design.
• if it's expensive, it probably won't be effective.
• Unless it is for your use alone you do not control what a system is to be used for
• Even if it is for your use alone you do not control the resources which will be pitted against your system
• cryptanalysis is difficult - but people can do difficult things
• people offering the impossible are lying
• in code, nothing ever really goes away

The principles can sometimes be broken or wrong, unlike the laws.

[2] the security of a secret is inversely proportional to the square of the number of people who know it

- Peter Fairbrother

0. It's all about who is in control
1. Someone else is after the things you have
2. An attacker can't steal things which aren't there to steal
3. Everywhere can be attacked
4. More complex systems provide more places to attack
5. Attack methods are many, varied, ever-changing and eternal
6. Only those you trust can betray you
7. Holes for good guys are holes for bad guys too
8. A system which is hard to use will be misused, abused and underused
9. Security is a Boolean. [1]
10. Items of data once publicly linked cannot be reliably unlinked

[1] Looking back in time from the future - did it work? Then it was secure enough. Can be hard to see that from the present though, and even from the future not all harmful breaches can be seen.

- Peter Fairbrother

Source: What Is PCIe Card? Everything You Need to Know About PCI Express Card

Take a look at Introduction to Linux Interrupts and CPU SMP Affinity. Controlling Core Assignments

ps aux |grep irq
irqbalance
cat /proc/interrupts

Algorithmic Trading Using Continuous Action Space Deep Reinforcement Learning

Price movement prediction has always been one of the traders' concerns in financial market trading. In order to increase their profit, they can analyze the historical data and predict the price movement. The large size of the data and complex relations between them lead us to use algorithmic trading and artificial intelligence.

This paper aims to offer an approach using Twin-Delayed DDPG (TD3) and the daily close price in order to achieve a trading strategy in the stock and cryptocurrency markets. Unlike previous studies using a discrete action space reinforcement learning algorithm, the TD3 is continuous, offering both position and the number of trading shares. Both the stock (Amazon) and cryptocurrency (Bitcoin) markets are addressed in this research to evaluate the performance of the proposed algorithm.

The achieved strategy using the TD3 is compared with some algorithms using technical analysis, reinforcement learning, stochastic, and deterministic strategies through two standard metrics, Return and Sharpe ratio. The results indicate that employing both position and the number of trading shares can improve the performance of a trading system based on the mentioned metrics.

Swarm of UAVs for Network Management in 6G: A Technical Review

Fifth-generation (5G) cellular networks have led to the implementation of beyond 5G (B5G) networks, which are capable of incorporating autonomous services to swarm of unmanned aerial vehicles (UAVs). They provide capacity expansion strategies to address massive connectivity issues and guarantee ultra-high throughput and low latency, especially in extreme or emergency situations where network density, bandwidth, and traffic patterns fluctuate. On the one hand, 6G technology integrates AI/ML, IoT, and blockchain to establish ultra-reliable, intelligent, secure, and ubiquitous UAV networks. 6G networks, on the other hand, rely on new enabling technologies such as air interface and transmission technologies, as well as a unique network design, posing new challenges for the swarm of UAVs. Keeping these challenges in mind, this article focuses on the security and privacy, intelligence, and energy-efficiency issues faced by swarms of UAVs operating in 6G mobile networks. In this state-of-the-art review, we integrated blockchain and AI/ML with UAV networks utilizing the 6G ecosystem. The key findings are then presented, and potential research challenges are identified. We conclude the review by shedding light on future research in this emerging field of research.

Perspectives on a 6G Architecture

Mobile communications have been undergoing a generational change every ten years. Whilst we are just beginning to roll out 5G networks, significant efforts are planned to standardize 6G that is expected to be commercially introduced by 2030. This paper looks at the use cases for 6G and their impact on the network architecture to meet the anticipated performance requirements. The new architecture is based on integrating various network functions in virtual cloud environments, leveraging the advancement of artificial intelligence in all domains, integrating different sub-networks constituting the 6G system, and on enhanced means of exposing data and services to third parties.

On Routing Optimization in Networks with Embedded Computational Services

Modern communication networks are increasingly equipped with in-network computational capabilities and services. Routing in such networks is significantly more complicated than the traditional routing. A legitimate route for a flow not only needs to have enough communication and computation resources, but also has to conform to various application-specific routing constraints. This paper presents a comprehensive study on routing optimization problems in networks with embedded computational services. We develop a set of routing optimization models and derive low-complexity heuristic routing algorithms for diverse computation scenarios. For dynamic demands, we also develop an online routing algorithm with performance guarantees. Through evaluations over emerging applications on real topologies, we demonstrate that our models can be flexibly customized to meet the diverse routing requirements of different computation applications. Our proposed heuristic algorithms significantly outperform baseline algorithms and can achieve close-to-optimal performance in various scenarios.

Network Intrusion Detection System in a Light Bulb

Internet of Things (IoT) devices are progressively being utilised in a variety of edge applications to monitor and control home and industry infrastructure. Due to the limited compute and energy resources, active security protections are usually minimal in many IoT devices. This has created a critical security challenge that has attracted researchers' attention in the field of network security. Despite a large number of proposed Network Intrusion Detection Systems (NIDSs), there is limited research into practical IoT implementations, and to the best of our knowledge, no edge-based NIDS has been demonstrated to operate on common low-power chipsets found in the majority of IoT devices, such as the ESP8266. This research aims to address this gap by pushing the boundaries on low-power Machine Learning (ML) based NIDSs. We propose and develop an efficient and low-power ML-based NIDS, and demonstrate its applicability for IoT edge applications by running it on a typical smart light bulb. We also evaluate our system against other proposed edge-based NIDSs and show that our model has a higher detection performance, and is significantly faster and smaller, and therefore more applicable to a wider range of IoT edge devices.

using ip link in brief mode:

# ip -br link
lo               UNKNOWN        00:00:00:00:00:00 
eno1             UP             c8:1f:66:ea:43:20 
eno2             UP             c8:1f:66:ea:43:21 
eno3             UP             c8:1f:66:ea:43:22 
eno4             DOWN           c8:1f:66:ea:43:23 

using ip addr in brief mode:

# ip -br a
lo               UNKNOWN        127.0.0.1/8 10.32.0.12/32 ::1/128
eno1             UP
eno2             UP
eno3             UP             10.32.3.12/24 fe80::ca1f:66ff:feea:4322/64
eno4             DOWN

Access to the SFP style:

# ethtool -m enp65s0f0np0
Identifier                                : 0x03 (SFP)
Extended identifier                       : 0x04 (GBIC/SFP defined by 2-wire interface ID)
Connector                                 : 0x23 (No separable connector)
Transceiver codes                         : 0x01 0x00 0x00 0x04 0x00 0x04 0x00 0x00 0x0d
Transceiver type                          : Infiniband: 1X Copper Passive
Transceiver type                          : Ethernet: 1000BASE-CX
Transceiver type                          : Passive Cable
Transceiver type                          : Extended: 25G Base-CR CA-N
Encoding                                  : 0x06 (64B/66B)
BR, Nominal                               : 25750MBd
Rate identifier                           : 0x00 (unspecified)
Length (SMF,km)                           : 0km
Length (SMF)                              : 0m
Length (50um)                             : 0m
Length (62.5um)                           : 0m
Length (Copper)                           : 1m
Length (OM3)                              : 0m
Passive Cu cmplnce.                       : 0x01 (SFF-8431 appendix E) [SFF-8472 rev10.4 only]
Vendor name                               : Mellanox
Vendor OUI                                : 00:02:c9
Vendor PN                                 : MCP2M00-A001E30N
Vendor rev                                : A4
Option values                             : 0x00 0x00
BR margin, max                            : 0%
BR margin, min                            : 0%
Vendor SN                                 : MT2010VB00194
Date code                                 : 200219
Optical diagnostics support               : No

driver and firmware info:

# ethtool -i enp65s0f0np0
driver: mlx5_core
version: 5.15.39-4-pve
firmware-version: 16.34.1002 (MT_0000000183)
expansion-rom-version:
bus-info: 0000:41:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes

Show ethernet controllers on the pci bus:

# lspci |grep -i ether
01:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
42:00.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
42:00.1 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]

Show details on a specific ethernet controller:

# lspci -vvnn -s 42:00.0
42:00.0 Ethernet controller [0200]: Mellanox Technologies MT27800 Family [ConnectX-5] [15b3:1017]
Subsystem: Mellanox Technologies MT27800 Family [ConnectX-5] [15b3:0061]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- 

Show a tree of entries (output is clipped):

~# lspci -tvnn
-+-[0000:7f]-+-08.0  Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Link 0 [8086:0e80]
|           +-09.0  Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Link 1 [8086:0e90]
.....

Show available physical slots and their capability:

# dmidecode -t 9
# dmidecode 3.3
Getting SMBIOS data from sysfs.
SMBIOS 2.7 present.

Handle 0x0900, DMI type 9, 17 bytes
System Slot Information
Designation: PCI1
Type: x8 PCI Express 3 x16
Current Usage: Available
Length: Long
ID: 1
Characteristics:
3.3 V is provided
PME signal is supported

Handle 0x0901, DMI type 9, 17 bytes
System Slot Information
Designation: PCI2
Type: x16 PCI Express 3
Current Usage: In Use
Length: Long
ID: 2
Characteristics:
3.3 V is provided
PME signal is supported
Bus Address: 0000:42:00.0

Handle 0x0902, DMI type 9, 17 bytes
System Slot Information
Designation: PCI3
Type: x16 PCI Express 3
Current Usage: Available
Length: Long
ID: 3
Characteristics:
3.3 V is provided
PME signal is supported

But what is actually negotiated might be different:

root@host01:~# lspci -vvnn -s 42:00.0|grep Lnk
LnkCap: Port #0, Speed 8GT/s, Width x16, ASPM not supported
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
LnkSta: Speed 5GT/s (downgraded), Width x16 (ok)
LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+ EqualizationPhase1+
LnkCtl3: LnkEquIntrruptEn- PerformEqu-

On one slot, I get 'LnkSta: Speed 8GT/s (ok), Width x8 (downgraded)' [8GT/s * x8 = 64], and in another slot it have "Speed 5GT/s (downgraded), Width x16 (ok)' [5GT/s * x16 = 80], so I guess we use x16 slot.

List kernel drivers for a specific pci slot:

# lspci -k -s 42:00
42:00.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
Subsystem: Mellanox Technologies MT27800 Family [ConnectX-5]
Kernel driver in use: mlx5_core
Kernel modules: mlx5_core
42:00.1 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
Subsystem: Mellanox Technologies MT27800 Family [ConnectX-5]
Kernel driver in use: mlx5_core
Kernel modules: mlx5_core

Show details for a specific kernel module:

# modinfo mlx5_core
filename:       /lib/modules/5.15.39-4-pve/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko
license:        Dual BSD/GPL
description:    Mellanox 5th generation network adapters (ConnectX series) core driver
author:         Eli Cohen 
srcversion:     8802C9A5BF30F219CD73715
alias:          auxiliary:mlx5_core.eth
....
alias:          auxiliary:mlx5_core.eth-rep
alias:          auxiliary:mlx5_core.sf
depends:        tls,pci-hyperv-intf,mlxfw,psample
retpoline:      Y
intree:         Y
name:           mlx5_core
vermagic:       5.15.39-4-pve SMP mod_unload modversions
parm:           debug_mask:debug mask: 1 = dump cmd data, 2 = dump cmd exec time, 3 = both. Default=0 (uint)
parm:           prof_sel:profile selector. Valid range 0 - 2 (uint)

• Understanding PCIe Configuration for Maximum Performance

Another additional lspci command combination for finding the kernel module for a particular pci connection:

~# lspci -knn -s 42:00.0
42:00.0 Ethernet controller [0200]: Mellanox Technologies MT27800 Family [ConnectX-5] [15b3:1017]
        Subsystem: Mellanox Technologies MT27800 Family [ConnectX-5] [15b3:0061]
        Kernel driver in use: mlx5_core
        Kernel modules: mlx5_core

The dummy interface is not used in the actual routing process in most cases. The dummy ip address is inserted into the local route table as a local address (check it out yourself: ip ro sh table local). Ingress packets from all interfaces are checked against this table first (as it is the maximum priority rule in the ip rule), and if a local-type route matches, the packet is queued for local delivery. Egress packets are even simpler: your default route will point towards your physical interface; the packets will be queued directly to that interface's queues and you'll keep all your regular GSOs and GROs -- at least the ones that are managed by the kernel.

There are standard ways many nics can offload packets: TSO (tcp segment offloading), gso (generic segment offloading), gro (generic receive offload), lro (large receive offload) are the common ones. You can see which ones are enabled with ethtool -k interface_name, interface usually has to be down to change them.