Raymond P. Burkholder - Things I Do - Containers

Image Building Notes - debootstrap

nospam@example.com (Raymond P. Burkholder) — Sun, 05 Apr 2026 17:31:32 +0000

APT based distributions like Debian can be containerized with a tool called debootstrap. It is part of the image build process of lxc-create. It is also referenced in Docker Base Images for building an image from scratch.

When looking at the build scripts included in the package installation, repositories for the following distributions can be found in /usr/share/debootstrap/scripts:

debian - universal operating system
trisquel - a distribution of the GNU operating system, with the kernel GNU Linux-libre
ubuntu - modern enterprise open source
pardus - Turkish
kali - open-source, Debian-based Linux distribution geared towards various information security tasks, such as Penetration Testing, Security Research, Computer Forensics and Reverse Engineering
elxr - Enterprise-Grade Linux for Edge-to-Cloud Deployments
pureos - fully-convergent, user friendly, secure and freedom respecting OS for your daily usage

The wiki shows a simple two liner to get the basics of the distribution in place (as root):

mkdir trixie-chroot
debootstrap stable trixie-chroot http://deb.debian.org/debian/

Enter the chroot and note new root:

root@test:~# pwd
/root
root@test:~# chroot trixie/
root@test:/# pwd
/
root@test:/# exit
exit
root@test:~# pwd
/root

After the debootstrap, create the base for docker, and give it a try:

tar -C trixie-chroot -c . | docker import - trixie
docker run trixie cat /etc/debian_version
docker run --rm -i -t trixie /bin/bash

Although debootstrap can be used to build an image for a version subsequent, it is generally recommended to use debootstrap from at least the desired version to ensure it has the proper updates and dependencies.

Command line to summarize the referenced repositories:

grep -h default_mirror /usr/share/debootstrap/scripts/* \
  | sed 's/default_mirror//' \
  | sed 's/[ \t]//g' \
  | sort \
  | uniq

LXC Fresh Container Construction From Scratch for Proxmox

nospam@example.com (Raymond P. Burkholder) — Fri, 27 Feb 2026 21:03:19 +0000

There are many articles available which discuss customizing a pre-existing Proxmox Container Template. Few, if any, discuss constructing an LXC container from scratch. Maybe because, fundamentally, a container template is just the rootfs as tarball, so building it is quite easy:

Build a linux based virtual machine, I use Debian's recent release
Install LXC and its template package
Construct and initialize an LXC container
Shut it down and and zip it up
Copy it over to the ProxMox template directory

The details:

# build the linux vm - details not relevant here
# ssh into the vm, or start a command line
# install basic packages

sudo apt install --no-install-recommends lxc lxc-templates xz-utils bridge-utils wget debootstrap rsync

# basic container templates are in:
#   /usr/share/lxc/templates/ 
# for debian as well as other distributions

# create an lxc container, provide a list any additional packages

lxc-create --template debian --name trixie-template -- --release trixie --packages iputils-ping,vim-tiny

# start and attach to the container
lxc-start trixie-template
lxc-attach trixie-template

# prepare for generating template
apt clean
apt purge

# Remove SSH host keys to ensure unique keys for each clone:
rm /etc/ssh/ssh_host_*

# Empty the machine ID file:
truncate -s 0 /etc/machine-id

# clear history
unset HISTFILE
# truncate history
history -c
> ~/.bash_history
# the following has a space in front to prevent inclusion in the history
 shutdown -h now

# the shutdown returns to the virtual machine's prompt
# compress the directory structure

cd /var/lib/lxc/trixie-template/

# remove /dev files as they can't be created in an unprivileged container
# an example error message if not removed:
#   tar: ./rootfs/dev/urandom: Cannot mknod: Operation not permitted
# construction of a new container will re-create the directory and files

rm ./rootfs/dev/ptmx
rm ./rootfs/dev/zero
rm ./rootfs/dev/tty3
rm ./rootfs/dev/urandom
rm ./rootfs/dev/null
rm ./rootfs/dev/tty
rm ./rootfs/dev/console
rm ./rootfs/dev/tty4
rm ./rootfs/dev/tty2
rm ./rootfs/dev/random
rm ./rootfs/dev/tty1
rm ./rootfs/dev/full

# cd into rootfs and zip the container

cd rootfs
tar --xz --acls --numeric-owner -cf /var/local/trixie-13-3-template.tar.xz ./

# the xz file can be copied over to proxmox and placed into
# /var/lib/pve/local-btrfs/template/cache/
# for use as a template for container creation

During the first use of lxc-create to create the original container, packages are downloaded and installed to build the container. The packages and installation is cached for faster subsequent builds of the same container type.

If the cache becomes stale, it can be rebuilt by using --flush-cache in a manner similar to:

lxc-create --template debian --name trixie-template -- --release trixie --flush-cache --packages iputils-ping,vim-tiny,less,python-minimal

An existing cache can be updated with something like:

sudo chroot /var/cache/lxc/debian/rootfs-trixie-amd64
apt-get update
apt-get dist-upgrade
apt-get clean
exit

courtesy of Updating lxc image/container caches

One other note, there are two package candidates for installing the ping utility:

iputils-ping - native Linux ping, preferred for Debian/Linux
inetutils-ping - general gnu version, used on a variety of posix sytstems, less preferred

Some fix-ups in the process:

apt-get install less
dpkg-reconfigure locales
useradd user

Talos Install in Preparation for Kubernetes

nospam@example.com (Raymond P. Burkholder) — Sun, 12 Jan 2025 17:35:54 +0000

On my ProxMox machine, I need a more dynamic way of handling containers rather than being limited to the LXC flavour offered by ProxMox. There was a question on reddit to which I supplied an answer about why to use full virtualization for Kubernetes vs running in a container. In summary:

proxmox is a virtualization engine which enables LXC containers and QEMU/KVM guest virtual machines
LXC containers are compartmentalized and share direct access to the kernel
containers are used so as not to pollute your based proxmox installation with more packages and runtimes
typically, you don't want to nest containers inside containers
virtual machines run their own kernel/operating system, and are more secure (failure/security) and independent
since k8s manages docker containers (not LXC containers), and you don't want to run containers of any fashion in containers of another fashion (docker inside lxc), you run a virtual machine with k8s to keep k8s packages and runtime separate from the core proxmox runtime environment
hence the container inside virtual machine on hypervisor platform
for security and compartmentalization

I built a basic LXC management container to take a look at talosctl, then used these instructions to perform a few tests:

apt update
apt upgrade
apt install --no-install-recommends curl
# based upon https://www.talos.dev/v1.9/talos-guides/install/talosctl/
curl -sL https://talos.dev/install | sh

# based upon https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256"
echo "$(cat kubectl.sha256)  kubectl" | sha256sum --check
install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client

I ultimately removed this container, as I started reading the Getting Started Guide. But the above did generate the information that the lastest talosctl version is v1.9.1, which is useful for generating the iso image required.

.... more to come

Reference:

Kubernetes Links

nospam@example.com (Raymond P. Burkholder) — Sat, 12 Nov 2022 22:43:54 +0000

What is Kubernetes and how should you monitor it? - A monitoring strategy that takes advantage of Kubernetes will give you a bird's eye view of your entire application’s performance, even if containers running your applications are continuously moving between hosts or being scaled up and down.
How to optimize Kubernetes resource configurations for cost and performance - in part two of this Kubernetes guide, you'll get help balancing appropriate parameter configuration for any cluster you are working with now or in the future. You'll learn about requests and limits, measuring CPU utilization, and how to optimize Kubernetes resource allocation.
Best way to install and use kubernetes for learning - reddit
Kubernetes The Hard Way - This tutorial walks you through setting up Kubernetes the hard way. This guide is not for people looking for a fully automated command to bring up a Kubernetes cluster. Kubernetes The Hard Way is optimized for learning, which means taking the long route to ensure you understand each task required to bootstrap a Kubernetes cluster.
Deploying a Kubernetes Cluster within Proxmox using Ansible

A Different Container Way

nospam@example.com (Raymond P. Burkholder) — Wed, 25 May 2022 16:23:52 +0000

Admin magazine has an article called Create secure simple containers with the systemd tools Nspawnd and Portabled, which offers a mechanism different from my favorite LXC mechanisms. I'll have to give it a try for simpler projects.

They seem to be subsets of the templates and caching already available via LXC, but the one stand out is an added tool called mkosi, which stands for Make Operating System Image, and is a tool for precisely that: generating an OS tree or image that can be booted. It seems to be useful for creating container images as well as images which can be used in heavier virtualization environments such as KVM.

IQFeed on Linux throws ICMP Error

nospam@example.com (Raymond P. Burkholder) — Tue, 26 May 2020 20:15:57 +0000

Seen on the IQFeed Developer Support 2020/04/21. A solution for allowing ICMP packets.

iqconnect.exe does a ping round trip against its DTN servers (you can see the ping results at IQConnect.log when you stop the feed and iqconnect.exe exits). Thing is, ping uses ICMP protocol, which in linux is somewhat privileged.
So, you need to give wine the appropiate permissions in order to be able to use ICMP. Running wine as root in order to circumvent this problem would be overkill (besides a very bad thing to do!), but fortunately you can use setcap in order to grant permissions in a much more granular way.
First, locate where your wine-preloader file is. In my case, it's on /usr/bin/wine-preloader . Then, type (yoo will need to sudo for this):
sudo setcap cap_net_raw+epi /usr/bin/wine-preloader
and that's all. Now wine is allowed to use ICMP protocol, which in turn will allow IQconnect.eze to make its "ping things" without complaining

GUI From an LXC Container on the Host

nospam@example.com (Raymond P. Burkholder) — Sat, 23 May 2020 21:41:27 +0000

I am almost embarrassed to say that I missed a good night's sleep sifting through erroneous out of date misinformation, missing some subtle distinctions, and winnowing out the chafe.

This all started when I wanted to give Krita a try for editing photographs. It is said it is the next best thing in open source when compared to PhotoShop. When installed on my Debian Linux workstation, all I could get out of it was crashes of one form or another. I'm not really surprised as my workstation has been through various combinations of buster, testing, bullseye, sid and experimental. Some package is out of sync somewhere.

So.. since I know how to run LXC containers, I figured I'd give that a try. That was successful, to a point. I used X2Go for remote console. But when my images are Nikon NEF files at 24Megapixels at 14bits each, file size, computation, and visualization are a bit of challenge (the screen updates being the main challenge). A wide erasure brush was slow, even on my speedy machine.

There are sites which vehemently say that there is no direct way to see the GUI from a container on a workstation host. Sigh. Misinformation. Then there are the five year old sites which show how it is done, but have extra commands, missing commands, or missing options. More sigh.

After much trial and error and trying the same things over again, with minor variations on the theme, in the hopes something might fix itself, it was a long night.

For the record, here is my research on a Debian Bullseye system with LXC '1:3.1.0+really3.0.4-3'
Continue reading "GUI From an LXC Container on the Host"

Limiting Memory in LXC Containers

nospam@example.com (Raymond P. Burkholder) — Tue, 01 Oct 2019 20:01:49 +0000

To limit the amount of memory supplied to each container, place something similar to the following in the container configuration file:

lxc.cgroup.memory.limit_in_bytes = 5120M

The lxcfs daemon/service/package should be installed, and should be seen to be mounted on /var/lib/lxcfs

Prior to setting the memory limit, unfettered memory useage can be determined with

cat /sys/fs/cgroup/memory/lxc//memory.max_usage_in_bytes

When inside the container, something like 'free -m' should show the memory available to the container.

# free -m
              total        used        free      shared  buff/cache   available
Mem:          16029        5121        5890         444        5017       10046
Swap:          9535           0        9535

Gui using Wine, X2Go, LXDE in a simple LXC container

nospam@example.com (Raymond P. Burkholder) — Mon, 02 Jul 2018 03:30:50 +0000

Skip this whole bit and go to the bottom for the current easier install.

Here are some instructions I use to build a light weight gui in an LXC container. The window easily scales, and with Wine, Windows applications can be run without the Windows overhead.

Using X2Go Packages for Debian GNU/Linux as a reference, I created /etc/apt/sources.list.d/x2go.list with the following content:

# X2Go Repository (release builds)
deb http://packages.x2go.org/debian stretch extras main
# X2Go Repository (sources of release builds)
deb-src http://packages.x2go.org/debian stretch extras main

I have the following primary sources in /etc/apt/sources.list:

deb [arch=amd64] http://ftp.us.debian.org/debian          stable        main contrib
deb [arch=amd64] http://security.debian.org/debian-security stable/updates main contrib
deb [arch=i386] http://ftp.us.debian.org/debian          stable        main contrib
deb [arch=i386] http://security.debian.org/debian-security stable/updates main contrib

Then the following commands will add wine64, wine32, lxde, and x2go server:

vim.tiny /etc/apt/sources.list
apt update
dpkg --add-architecture i386 && apt-get update && apt-get install wine32 wine winetricks
apt update
apt --no-install-recommends install gnupg2 dirmngr
apt-key adv --recv-keys --keyserver keyserver.ubuntu.com E1F958385BFE2B6E
vim.tiny  /etc/apt/sources.list.d/x2go.list
apt update
apt-get install x2go-keyring && apt-get update
apt install \
  --no-install-recommends \
    xserver-xorg-core \
    xserver-xorg-input-all \
    xserver-xorg-video-fbdev  \
    xserver-xorg-video-vesa \
    lxde \
    x2goserver-xsession \
    x2golxdebindings

VNC client is not used. An X2Go client is used instead. It seems to be fast and efficient. And works through ssh. It can even do a proxied hop through one ssh connection to a second ssh destination.

Current Easier Install, everything has been packaged now:

apt install \
  --no-install-recommends \
    xserver-xorg-core \
    xserver-xorg-input-all \
    xserver-xorg-video-fbdev  \
    xserver-xorg-video-vesa \
    lxde \
    x2goserver \
    x2goserver-xsession

Philosophy of Containers

nospam@example.com (Raymond P. Burkholder) — Sun, 18 Feb 2018 16:45:30 +0000

Lifted from LWN [Posted February 10, 2018 by jake] -- In ACMQueue magazine, Bridget Kromhout writes about containers and why they are not the solution to every problem. The article is subtitled: "Complex socio-technical systems are hard; film at 11."

Don't get me wrong—containers are delightful! But let's be real: we're unlikely to solve the vast majority of problems in a given organization via the judicious application of kernel features. If you have contention between your ops team and your dev team(s)—and maybe they're all facing off with some ill-considered DevOps silo inexplicably stuck between them—then cgroups and namespaces won't have a prayer of solving that. Development teams love the idea of shipping their dependencies bundled with their apps, imagining limitless portability. Someone in security is weeping for the unpatched CVEs, but feature velocity is so desirable that security's pleas go unheard. Platform operators are happy (well, less surly) knowing they can upgrade the underlying infrastructure without affecting the dependencies for any applications, until they realize the heavyweight app containers shipping a full operating system aren't being maintained at all.

The original article is found at Containers Will Not Fix Your Broken Culture (and Other Hard Truths) -- Complex socio-technical systems are hard; film at 11.

My current role revolves around automating the build of solutions: operating systems, the networking, the virtualization, the storage and the apps running on top. And not just home grown software modules. So the above quote struck a chord. And is reinforced by another paragraph in the article:

Being able to reproduce a build allows for separation of concerns. We want this to be effective and yet not introduce unnecessary barriers. The proverbial wall of confusion is all too real, built on the tension between having incentive to ship changes and being rewarded for stability. Building just the right abstractions that empower independent teams is worth taking the time to iterate on (and, no, nobody gets it right immediately, because "right" will evolve over time).

While on the subject of containers, there was another recent LWN reference: Containers from user space in which Jonathan Corbet writes about Jessie Frazelle's talk at linux.conf.au 2018. The article reminds me that Seccomp, Apparmour and SELinux are important tools for enhancing the 'compartmentalization' of containers.

Containers, when used by developers for publishing their apps and dependencies, could be defined as a form of packaging. This definition segues into an article linked from Planet Debian which is called futures of distributions where the author makes reference to the issue:

The granularity at which software is built has fundamentally changed. It's now typical for hundreds of small libraries to be used by any application, often pegged to specific versions. Language-specific tools manage all the resulting complexity automatically, but distributions can't muster the manpower to package a fraction of this stuff.
2018/02/19 - even after saying all that, pre-built containers are here to stay: OpenFaaS -- you can package anything as a serverless function - from Node.js to Golang to CSharp, even binaries like ffmpeg or ImageMagick. I read somewhere that serverless is just something you can't ssh into. So there is a wall be built between those who build the underlying infrastructure and those who put the window dressing on. And is confirmed by OpenFaaS moto:
Our core values are: developer first, operational simplicity and community centric

Not saying serverless is good, bad, or ugly, but someone has a Curated list of resources related to serverless architectures and the Serverless Framework.

LXC Container Research Items

nospam@example.com (Raymond P. Burkholder) — Sun, 29 Oct 2017 12:06:47 +0000

Memory inside Linux containers: unanswered questions about how to check memory utilization within a container rather than the machine
LXC 1.0: GUI in Containers by Stephane Graber, notes about non root containers, and running skype in a container.

cirros: a tiny container base

nospam@example.com (Raymond P. Burkholder) — Mon, 09 Oct 2017 02:21:44 +0000

I havn't tried it out yet, but from somewhere, I found cirros: CirrOS is a Tiny OS that specializes in running on a cloud.

Some documentation show how to get it into an lxc container.

Should run as a really tiny kvm?

I found in a Debian install that CirrOS is an LXC template (with the install of the LXC package):

# ls -alt /usr/share/lxc/templates/lxc-cirros 
-rwxr-xr-x 1 root root 10374 Aug 25 18:20 /usr/share/lxc/templates/lxc-cirros

Container Analysis

nospam@example.com (Raymond P. Burkholder) — Sun, 23 Oct 2016 20:03:50 +0000

An interesting repository encountered today:

cinf: A command line tool to view namespaces and cgroups, useful for low-level container prodding.