Entries by Raymond P. Burkholder

From Things you can do with Debian: multimedia editing

• gimp - image processing
• audacity - quick audio recording / editing
• ardour - audio recording / editing / mixing / mastering
• kdenlive - video editing / mixing
• openshot - video editing / mixing
• handbrake - video transcoding

Simple streaming the Linux desktop to Kodi using GStreamer and RTP by Petter Reinholdtsen

2018/09/13 - Distributing static routes with DHCP

Intel has released a new NUC, the NUC8i7HVK. It used an integrated AMD GPU graphics engine, the Radeon RX Vega M GH in the NUC8i7HVK model.

During the initial release days, most of the installation and configuration documentation revolved around installing a specific version of Ubuntu, and installing a particular driver set from the AMD support pages.

That is no longer necessary. AMD has been contributing a significant amount of code to recent versions of the kernel for driving their new graphics chipsets.

I had tried to load, build, and install the amdgpu-pro driver, but it would never match up with the current kernel header files in Stretch and Buster.

One important note: the built in Intel graphics adapter needs to be disabled in the BIOS. Then these are the installation steps to get things running (I have two 4K screens attached, both rotated 90 degrees). So things seem to work.

In addition, i have the following in /etc/default/grub (run update-grub after changing):

GRUB_CMDLINE_LINUX="amdgpu.dc=1"

I used the non-free firmware image of Debian Buster to get the ball rolling. I think I encountered some black screens so had to ssh in from a different computer to fix various gaffes on the way to getting something working.

apt -t experimental install linux-image-4.18.0-rc4-amd64 linux-headers-4.18.0-rc4-amd64
reboot
apt -t experimental install mesa-utils mesa-utils-extra mesa-va-drivers \
         mesa-vdpau-drivers mesa-vulkan-drivers libglx-mesa0 \
         xserver-xorg-video-amdgpu
wget -m -np https://people.freedesktop.org/~agd5f/radeon_ucode/vegam/
mkdir /lib/firmware/amdgpu
cp people.freedesktop.org/~agd5f/radeon_ucode/vegam/*.bin /lib/firmware/amdgpu/
/usr/sbin/update-initramfs -u -k all
reboot

Which results in:

# lspci|grep -i vga
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Polaris 22 [Radeon RX Vega M GH] (rev c0)
# uname -a
Linux nuc8i7hvk01 4.18.0-rc4-amd64 #1 SMP Debian 4.18~rc4-1~exp1 (2018-07-12) x86_64 GNU/Linux

Long story short, the solution was to install the latest firmware. In this case, I have an Intel NUC8i7HVK Mini PC with Intel Wireless-AC 8265 and Bluetooth 4.2. I installed the Intel wireless firmware package from experimental:

apt -t experimental install firmware-iwlwifi

This is also running current experimental kernel:

# uname -a
Linux nuc8i7hvk01 4.18.0-rc4-amd64 #1 SMP Debian 4.18~rc4-1~exp1 (2018-07-12) x86_64 GNU/Linux

Which provided a more recent load on boot:

[ 2.478527] iwlwifi 0000:06:00.0: firmware: direct-loading firmware iwlwifi-8265-36.ucode

Upon reboot, it was easy to scan, connect, and pair the Logitech MX Anywhere 2S mouse.

For future reference, here are some commands to see some activiites:

$ hcitool dev
Devices:
        hci0    0C:54:15:66:3B:9E

$ hciconfig
hci0:   Type: Primary  Bus: USB
        BD Address: 0C:54:15:66:3B:9E  ACL MTU: 1021:4  SCO MTU: 96:6
        UP RUNNING PSCAN ISCAN
        RX bytes:41366 acl:2102 sco:0 events:164 errors:0
        TX bytes:4913 acl:29 sco:0 commands:75 errors:0

# bluetoothctl
Agent registered
[MX Anywhere 2S]# help
Menu main:
Available commands:
-------------------
advertise                                         Advertise Options Submenu
scan                                              Scan Options Submenu
gatt                                              Generic Attribute Submenu
list                                              List available controllers
show [ctrl]                                       Controller information
select                                      Select default controller
devices                                           List available devices
paired-devices                                    List paired devices
system-alias                                Set controller alias
reset-alias                                       Reset controller alias
power                                     Set controller power
pairable                                  Set controller pairable mode
discoverable                              Set controller discoverable mode
agent                          Enable/disable agent with given capability
default-agent                                     Set agent as the default one
advertise                            Enable/disable advertising with given type
set-alias                                  Set device alias
scan                                      Scan for devices
info [dev]                                        Device information
pair [dev]                                        Pair with device
trust [dev]                                       Trust device
untrust [dev]                                     Untrust device
block [dev]                                       Block device
unblock [dev]                                     Unblock device
remove                                       Remove device
connect                                      Connect device
disconnect [dev]                                  Disconnect device
menu                                        Select submenu
version                                           Display version
quit                                              Quit program
exit                                              Quit program
help                                              Display help about this program

[MX Anywhere 2S]# paired-devices
Device E1:38:CB:49:A9:A5 MX Anywhere 2S

[MX Anywhere 2S]# devices
Device E1:38:CB:49:A9:A5 MX Anywhere 2S

[MX Anywhere 2S]# paired-devices
Device E1:38:CB:49:A9:A5 MX Anywhere 2S

Debian wiki on bluetooth: Using Bluetooth in a Debian system

• XAGYL Communications: well stocked Ontario Mikrotik distribution
• Solimedia Technology Inc.: Vancouver based online ordering, with some interesting 4G and IP Telephony products
• Baltic Networks: US based distribution with larger product line
• Miro: an African company with product lines of reference interest
• MikroTik Router Hardening
• automated backup scripts
• Mikrotik Open Ports which may need firewalling
• Tools for effortless reverse engineering of MikroTik routers
• SFP module compatibility table
• Managing 1500 routers with a single click: high level notes only (I'll probably use SaltStack with the SSH proxy capability instead)
• Manual:Flashfig
• Automatically provisioning RouterBoards: background info with someone's specific ideas on a domain specific language
• Manual:CAPsMAN: Controlled Access Point system Manager (CAPsMAN) allows centralization of wireless network management and if necessary, data processing
• CPE WAN Management Protocol & Auto Configuration Server from a TR-069 perspective but Mikrotik NetInstall is still integral to a clean recover process
• automatic backup '/system scheduler' ideas
• Mikrotik Script Repository

• Linux Kernel Archives
• Linux Kernel Mailing List
• Kernel Newbies
• LWN
• phoronix
• Linux Compatible

Other randomness:

• Networking CAKE Is Ready For Tasting With Linux 4.19: CAKE Qdisc (exposed as the sch_cake module) aims to "squeeze the most bandwidth and latency out of even the slowest ISP links and routers". The comments section has a number of links to other resources.
• Clixon is an automatic configuration manager where you generate interactive CLI, NETCONF, RESTCONF and embedded databases with transaction support from a YANG specification.
• YANG-Based Unified Modular Automation Tools
• Infrared temperature sensor driver for Linux kernel released - not sure how useful this is but leads to other products of possible value

Prelude To A 2008 Event: Paper Gold Manipulation Intensifies

And has anyone checked gold lease rates lately? Currently the lease rate curve for gold and silver in London is inverted. In fact, lease rates gold from 3 months to a year are negative. Negative lease rates mean the Central Banks will pay bullion banks to lease gold and silver. Long-timers like me know that this means there’s an immediate and anticipated shortage of physical gold and silver available for delivery, where “delivery” means the metal is removed from the London vaults and shipped to the entitled buyer.

Both gold and silver are backwardated. It took 11 iterations in the LBMA p.m. fix on Tuesday to balance out the heavy demand for physical gold from bidders. 11 iterations is rare occurrence. 5-6 iterations is rare. 1 or 2 is typical. Metal is tight in London.

If you are monitoring the Comex Hong Kong kilo bar vaults, you are aware that the movement in and out of the vaults there suggests that metal is also tight in Hong Kong, which means it is likely tight in Shanghai.

• PacketLife's Armory - encryption, exploitation, intrusion detection, forensics, vulnerability scanning, system monitoring, packet crafting, spoofing, ...
• IpCisco's Network Tools - packet analyzers, security audit, simulators, intrusion detection, ...

Here are some instructions I use to build a light weight gui in an LXC container. The window easily scales, and with Wine, Windows applications can be run without the Windows overhead.

Using X2Go Packages for Debian GNU/Linux as a reference, I created /etc/apt/sources.list.d/x2go.list with the following content:

# X2Go Repository (release builds)
deb http://packages.x2go.org/debian stretch extras main
# X2Go Repository (sources of release builds)
deb-src http://packages.x2go.org/debian stretch extras main

I have the following primary sources in /etc/apt/sources.list:

deb [arch=amd64] http://ftp.us.debian.org/debian          stable        main contrib
deb [arch=amd64] http://security.debian.org/debian-security stable/updates main contrib
deb [arch=i386] http://ftp.us.debian.org/debian          stable        main contrib
deb [arch=i386] http://security.debian.org/debian-security stable/updates main contrib

Then the following commands will add wine64, wine32, lxde, and x2go server:

vim.tiny /etc/apt/sources.list
apt update
dpkg --add-architecture i386 && apt-get update && apt-get install wine32 wine winetricks
apt update
apt --no-install-recommends install gnupg2 dirmngr
apt-key adv --recv-keys --keyserver keys.gnupg.net E1F958385BFE2B6E
vim.tiny  /etc/apt/sources.list.d/x2go.list
apt update
apt-get install x2go-keyring && apt-get update
apt install \
  --no-install-recommends \
    xserver-xorg-core \
    xserver-xorg-input-all \
    xserver-xorg-video-fbdev  \
    xserver-xorg-video-vesa \
    lxde \
    x2goserver-xsession \
    x2golxdebindings

VNC client is not used. An X2Go client is used instead. It seems to be fast and efficient. And works through ssh. It can even do a proxied hop through one ssh connection to a second ssh destination.

Q.  Which is the greater problem in the world today,
    ignorance or apathy?
A.  I don't know and I couldn't care less.

"If was one thing all people took for granted, was conviction that if you 
 feed honest figures into a computer, honest figures come out. Never doubted 
 it myself till I met a computer with a sense of humor."
                             Robert A. Heinlein, The Moon is a Harsh Mistress

hope for the best, plan for the worst

Never assume malice ("on purpose") something which can be adequately explained by incompetance.

• rtl-sdr - an ultra cheap software defined radio based on DVB-T TV tuners with RTL2832U chips. The RTL-SDR can be used as a wide band radio scanner. It may interest ham radio enthusiasts, hardware hackers, tinkerers and anyone interested in RF.
• arch daily - to collect the most important information to - help architects create better architecture, became a fast growing technology company that delivers inspiration, tools and knowledge to the 10 million architects that visit and use ArchDaily each month.
• Migrating to open source software: Linux - Going Open Source: An Overview of Open Source Software and Linux Migration

On a Debian Stretch KDE system, some combination of the following was able to change the natural scrolling direction to match that of a MacOS computer.

apt install xinput
apt install xserver-xorg-input-synaptics
apt install xserver-xorg-input-evdev

Place the following (from /usr/share/X11/xorg.conf.d/70-synaptics.conf) in /etc/X11/xorg.conf.d/70-synaptics.conf:

# Example xorg.conf.d snippet that assigns the touchpad driver
# to all touchpads. See xorg.conf.d(5) for more information on
# InputClass.
# DO NOT EDIT THIS FILE, your distribution will likely overwrite
# it when updating. Copy (and rename) this file into
# /etc/X11/xorg.conf.d first.
# Additional options may be added in the form of
#   Option "OptionName" "value"
#
Section "InputClass"
        Identifier "touchpad catchall"
        Driver "synaptics"
        MatchIsTouchpad "on"
# This option is recommend on all Linux systems using evdev, but cannot be
# enabled by default. See the following link for details:
# http://who-t.blogspot.com/2010/11/how-to-ignore-configuration-errors.html
#       MatchDevicePath "/dev/input/event*"
        Option "NaturalScrolling" "true"
EndSection

Section "InputClass"
        Identifier "touchpad ignore duplicates"
        MatchIsTouchpad "on"
        MatchOS "Linux"
        MatchDevicePath "/dev/input/mouse*"
        Option "Ignore" "on"
EndSection

# This option enables the bottom right corner to be a right button on clickpads
# and the right and middle top areas to be right / middle buttons on clickpads
# with a top button area.
# This option is only interpreted by clickpads.
Section "InputClass"
        Identifier "Default clickpad buttons"
        MatchDriver "synaptics"
        Option "SoftButtonAreas" "50% 0 82% 0 0 0 0 0"
        Option "SecondarySoftButtonAreas" "58% 0 0 15% 42% 58% 0 15%"
EndSection

# This option disables software buttons on Apple touchpads.
# This option is only interpreted by clickpads.
Section "InputClass"
        Identifier "Disable clickpad buttons on Apple touchpads"
        MatchProduct "Apple|bcm5974"
        MatchDriver "synaptics"
        Option "SoftButtonAreas" "0 0 0 0 0 0 0 0"
EndSection

In 'System Settings' -> 'Input Devices' -> TouchPad -> Scrolling, I have 'Reverse Scrolling' -> Vertical enabled.

The last section of SynapticsTouchpad mentioned the driver to install, which contradicts the beginning of the page, but indicates that libinput lacks what synaptics provides, so the roles need to be reversed.

From [SOLVED] KDE reverse/natural scrolling not working, I added the Option "NaturalScrolling" "true" line in the above file.

Reverse two finger scroll direction (natural scrolling)? didn't help much, but did introduce the "xinput" command.

I rebooted to get everything functional.

A place to keep links for embedded systems components, parts, robotics, ...

• Olimex - European company with the OLinuXino brand, and can be found through various distribution channels.
• seeed studio - IoT hardware enabler
• AdaFruit - to create the best place online for learning electronics and making the best designed products for makers of all ages and skill levels
• Think Penguin - not necessarily embedded, but they do provide Linux based computers and laptops, linked from TCP small queues and WiFi aggregation — a war story.
• Digilent embedded systems of various styles, fpga, ...

On 06/14/2018 09:22 PM, someone wrote:
> So I have to ask, why is it advantageous to put this in a container 
> rather than just run it directly
> on the container's host? 

Most any host now-a-days has quite a bit of horse power to run services. All those services could be run natively all in one namespace on the same host, or ...

I tend to gravitate towards running services individually in LXC containers. This creates a bit more overhead than running chroot style environments, but less than running full fledged kvm style virtualization for each service.

I typically automate the provisioning and the spool up of the container and its service. This makes it easy to rebuild/update/upgrade/load-balance services individually and enmasse across hosts.

By running BGP within each container, BGP can be used to advertise the loopback address of the service. I go one step further: for certain services I will anycast some addresses into bgp. This provides an easy way to load balance and provide resiliency of like service instances across hosts,

Therefore, by running BGP within the container, and on the host, routes can be distributed across a network with all the policies available within the bgp protocol. I use Free Range Routing, which is a fork of Quagga, to do this. I use the eBGP variant (RFC 7938) for the hosts and containers , which allows for the elimination of the extra overhead of OSPF or similar internal gateway protocol.

Stepping away a bit, this means that BGP is used in a tiered scenario. There is the regular eBGP with the public ASN for handling DFZ-style public traffic. For internal traffic, private eBGP ASNs are used for routing traffic between and within hosts and containers.

With recent improvements to Free Range Routing and the Linux Kernel, various combinations of MPLS, VxLAN, EVPN, and VRF configurations can be used to further segment and compartmentalize traffic within a host, and between containers. It is now very easy to run vlan-less between hosts through various easy to configure encapsulation mechanisms. To be explicit, this relies on and contributes to a resilient layer 3 network between hosts, and eliminates the bothersome layer 2 redundancy headaches with spanning tree and such.

That was a very long winded way to say: keep a very basic host configuration running a minimal set of functional services, and re-factor the functionality and split it across multiple containers to provide easy access to and maintenance of individual services like dns, smtp, database, dashboards, public routing, private routing, firewalling, monitoring, management, ...

There is a higher up-front configuration cost, but over the longer term, if configured via automation tools like Salt or similar, maintenance and security is improved.

It does require a different level of sophistication with operational staff.

Other BGP routing Daemons:

• exabgp - python based
• gobgp - go based
• bird - c based
• bagpipe - python based, in openstack project

Other mailing list comments:

Our use case was both on exporting service IPs as well as receiving routes from ToRs. Exa is more geared towards the former than the latter. Rather then working on getting imports and route installation through Exa, we found it simpler with BIRD exporting the service IP from it bound to a loopback to run local healthchecks on the nodes and then have them yank the service IP from the loopback on failing healthchecks in order to stop exporting.

The intent of the original post was vague. Like a lot of people, I would not run a full BGP router in a container. Now, if the purpose is to inject or learn a handful of routes in order to do limited host routing, I can see the need. A route-server or a looking glass in a container would be fine, or something to perform analysis on the routing table, but not anything that has to route actual traffic.

I use ExaBGP to inject routes, perfect tool for that. If routes have to be received (not my use case) it makes more sense, as stated by previous posts, to use Quagga or BIRD. Which one is better : easy : if you like Cisco better, use Quagga. If you like Juniper better, use BIRD

BIRD looking glass looks very good

Using 5 Commands for Checking Memory Usage in Linux as a starting point, here is a summary of commands:

• top - use '<' and '>' to select different columns to sort
• free - to report in MB and to sum the columns: free -mt
• vmstat - 'vmstat -s' to provide labelled statistics, 'vmstat 2 5' - run five times at 2 second intervals, 'vmstat -d' for disk statistics
• dmidecode - 'dmidecode -t' to find types, 'dmidecode -t memory' shows memory modules, 'dmidecode -t memory|grep -i size' shows slots and how they are filled
• cat /proc/meminfo - 'egrep --color 'Mem|Cache|Swap' /proc/meminfo' to select and colorize entries

The reason to use these commands is my monitoring system says shared memory usage is increasing. The following command should show something, but doesn't:

# ipcs -m

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status

# cat /proc/sysvipc/shm
       key      shmid perms                  size  cpid  lpid nattch   uid   gid  cuid  cgid      atime      dtime      ctime                   rss                  swap

# ipcs -a

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status

------ Semaphore Arrays --------
key        semid      owner      perms      nsems

# ipcs -lm

------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 18014398509465599
max total shared memory (kbytes) = 18014398442373116
min seg size (bytes) = 1

Maybe it doesn't matter. From top:

The amount of shared memory used by a task. It simply reflects memory that could be potentially shared with other processes.

A look behind the scenes on high frequency trading data:

• From 2012: TradingPhysics Historical TotalView-ITCH Files - a one post blog with some examination of a post mortem order book.
• QuantInsti’s Blog on Algo Trading and Quantitative Finance
• No BS Day Trading - training materials
• Idiots Guide to High Frequency Trading - Mark Cuban, some philosophy