- mounting a qemu image: short tutorial uses the nbd module, along with qemu-nbd to mound and view a qcow2 image
- QEMU/Images discusses image types, creating an image, converting images between popular formats, copying an image to physical device for use, when to use multiple images, how to share images, additional info on mounting, as well as obtaining information about the image.
- Virtualization Administration Guide: the Fedora guide, but generic enough to be used on many distributions. Impressive. And covers libvirt tools. With examples from virt-maanger (the gui).
Sunday, July 23. 2017
kvm qemu nbd
Monday, July 3. 2017
Linux Virtualization
Link to a Planet Virt Tools which is a collection of bloggers talking about Linux Virtualisation.
To go along with that is an amazing collection of Debian Build Tools, which includes things like debootstrap, qemubuilder, debian-installer, virt-inst, debootstick, Kameleon and a bunch of others.
Buiding Debian Images talks about various ways of building debian based images. The slide show mentions bootstarp-vz as the way many providers build their images.
Monday, April 10. 2017
KVM / QEMU Bits n Pieces
I don't think I've landed here before: QEMU Emulator User Documentation. It references 2.10 Direct Linux Boot for fast Linux Kernel testing. The documentation is referenced from QEMU: processor emulator.
libVirt is a virtualization API.
Daniel P. Berrangé is a prolific writer on all things KVM / QEMU / Virtualization / libvirt.
While using Planet Virt Tools as a jumping off point for 'blogging about open source virtualization", I came across Virtio 1.0 and Windows Guests.
Last but not least, we have the KVM - Kernel Virtual Machine.
Friday, October 28. 2016
Automating the Vagrant Guest Additions Build
In an earlier post, I wrote about installing Vagrant Guest Additions in a Debian Stretch distribution.
Here is a script, which needs to be run as root inside the guest to be used as the Vagrant package base, which will perform the necessary changes. For running with Linux kernel 4.8 or later, VirtualBox 5.1.8 or later will be required.
The command line takes one parameter... the target VirtualBox version, which at the time of this writing, is 5.1.8. This example would be run as root with (if the code is in a file called additions.sh):
bash additions.sh 5.1.8
The script takes care of obtaining the GuestAdditions iso, installing related packages, building the additions, removing the packages, and zeroing out dead disk space. This makes it ready for the second script which will generate the package.box.
#! /usr/bin/bash
function cleanup {
# package clean up
apt-get autoremove
apt-get purge
apt-get clean
apt-get autoclean
# remove indexes
rm /var/lib/apt/lists/*
# todo: look at /lib/modules when kernel upgraded
# zero out unused space for more efficient packing
echo "Zeroing out dead space ..."
dd if=/dev/zero of=/EMPTY bs=1M
rm -f /EMPTY
}
function vagrantuser {
# sudo settings
cat <> /etc/sudoers.d/10-sudo-overrides
Defaults env_keep += "GIT_*"
Defaults>root env_keep += SSH_AUTH_SOCK
Defaults>root !requiretty
%sudo ALL=NOPASSWD: ALL
EOT
# default vagrant key for bootstrapping
echo "Installing default key ...."
mkdir /home/vagrant/.ssh
chmod 700 /home/vagrant/.ssh
pushd /home/vagrant/.ssh
wget --no-check-certificate \
'https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub' \
-O authorized_keys
chmod 600 /home/vagrant/.ssh/authorized_keys
chown -R vagrant /home/vagrant/.ssh
popd
}
function guestadditions {
VBOXVER="$1"
# obtain and mount the VBoxGuestAdditions iso
echo "Obtaining Guest Additions ..."
wget \
http://download.virtualbox.org/virtualbox/${VBOXVER}/VBoxGuestAdditions_${VBOXVER}.iso
mkdir /media/VBoxGuestAdditions
mount -o loop,ro VBoxGuestAdditions_${VBOXVER}.iso /media/VBoxGuestAdditions
# build the additions
echo "Building additions ..."
KERN_DIR=/usr/src/linux-headers-$(uname -r)/ \
sh /media/VBoxGuestAdditions/VBoxLinuxAdditions.run
# unmount and remove the iso
echo "Building done, unmount iso ..."
umount /media/VBoxGuestAdditions
rmdir /media/VBoxGuestAdditions
rm VBoxGuestAdditions_${VBOXVER}.iso
}
function build {
VBOXVER="$1"
echo "sources: "
cat /etc/apt/sources.list | awk '/^deb/ {print $0}'
# sudo and ssh key are one time settings
if [[ "" == "$(ls -d /usr/src/vbox*)" ]]; then
vagrantuser
fi
# install build tools for the VirtualBox Guest Additions, required for Vagrant
echo "Installing packages ..."
apt-get update
PACKAGES=$(apt-get -y install linux-headers-$(uname -r) build-essential | awk \
'BEGIN { flag=0; list="" } \
/^[^ ]/ { flag=0 } \
/NEW packages will be installed/ { flag=1 } \
/ / { if (1 == flag) list=list$0 } \
END { print list }' \
)
echo "Installed packages: ${PACKAGES}"
guestadditions ${VBOXVER}
# remove packages
echo "Removing packages and cleaning up ..."
apt-get -y remove ${PACKAGES}
apt-get -y remove linux-headers-$(uname -r)
# add in some commonly used packages
apt-get -y install less tcpdump vim
cleanup
# show guest stuff installed
ls -alt /usr/src
df -h
IMAGECNT=$(dpkg -l | grep linux-image | wc -l)
if [ "1" != "${IMAGECNT}" ]; then
echo "More than one image installed, remove excess to reclaim space"
dpkg -l | grep linux-image
fi
}
if [ "root" != $(whoami) ]; then
echo "need to be user root"
else
if [ "" == "$1" ]; then
echo "[virtualbox version code (like 5.1.
| clean]"
else
case "$1" in
clean)
cleanup
;;
*)
build $1
;;
esac
fi
fi
Ensure the VirtualBox guest network interface #1 is NAT with para-virtualized IO. And verify the CD is removed.
Here is a script to run to pack a VirtualBox VM/Template into a box file for use by Vagrant (after the above script has been applied inside of the guest). On Windows, I run it in a cygwin environment. On Linux, it can be run natively.
#! /usr/bin/bash
# $1 - source base name
# $2 - packaged box name
if [ 2 != $# ]; then
echo "two arguments: VirtualBoxVMName PackagedBoxName"
else
VIRTUALBOXNAME=$1
PACKAGEDNAME=$2
# create a basic description file
cat <<EOT >> ${PACKAGEDNAME}.vf
Vagrant::Config.run do |config|
config.ssh.username = "vagrant"
end
EOT
# package up the guest
vagrant package --base ${VIRTUALBOXNAME} --vagrantfile ${PACKAGEDNAME}.vf
if [[ ! -d boxes ]]; then
mkdir boxes
fi
if [[ -e boxes/${PACKAGEDNAME}.box ]]; then
rm boxes/${PACKAGEDNAME}.box
fi
mv package.box boxes/${PACKAGEDNAME}.box
rm ${PACKAGEDNAME}.vf
fi
Friday, October 23. 2015
VirtualBox Based Virtual Machines With More Than 4 Interfaces From The Command Line
I wanted to test a multi-machine solution in VirtualBox, but I was brought up short because the user interface only allows the creation of four network interfaces. I needed 7 interfaces on some of the machines I was created.
The solution is to use VirtualBox's VBoxManage.exe application to create virtual machines from the command line. And once I figured out how to create virtual machines from the command line, I find it easy to create a number of machines, all with consistent properties. Or if some machines require special properties, it becomes easy to adjust the recipe on an individual virtual machine basis. And the if a property needs adjusting, it becomes easy to delete and rebuild the machine.
By putting all the properties into a script file, and then running the file, the properties are updated in the user interface, right before your eyes, as each property is adjusted. This certainly removes a bunch of repetitive point and click. And makes for a reproducible environment.
Read the full article to see the examples. Continue reading "VirtualBox Based Virtual Machines With More..." »
Monday, August 3. 2015
VMware, SDK, C++, Python, and Soap
- Problems connecting to VI SDK using C++/gsoap-2.7
- gSOAP Toolkit Software Download Instructions
- Need start up with VMWare API in c++
- Issues compiling vmware web api client code into useable objects
- non-proprietary Web services specifications
- vsphere api to change ip address of VirtualMachine
- vSphere Web Services SDK Documentation
- geeklee's notes on pyVmomi
- pyvmomi login to guest example
- VMware Infrastructure Object Model Overview
- Comparison of Python VMware vSphere client libraries
- vSphere Web Services SDK Programming Guide
- pyVmomi, finally ! (vSphere python SDK)
- pyvmomi/sample/getvmstree.py
Friday, July 24. 2015
KVM, LXC Containers and Networking
Policy Based Routing: using named routing table in Linux, a basic outline:
With Quagga, no. But you can implement it in the Linux shell. Examples from Ubuntu: 1. First create a named routing table in /etc/iproute2/rt_tables 200 MY-TABLE 2. Then create routes in that table: ip route add default via 172.16.0.1 table MY-TABLE 3. Finally, create the policy: ip rule add from 10.0.0.0/29 table MY-TABLE See all the "ip rule" commands for full details. I do a lot of this.
Tuesday, April 7. 2015
Docker on Debian Bits and Pieces
I'm in the process of figuring out Docker, and how to make it run on Debian. These are just 'notes to self' to look at for another time. I am reverting to lxc (linux containers) for the time being.
- Debian package for Docker is still several versions behind (with the current appearing to be v1.50 from 2015/02/03, see Docker release notes). It is better to download the binary from Docker.
- Some notes using Docker in a multi-host environment: Multi-host Docker and OpenvSwitch
- Some web sites suggest 'apt-get install docker.io' on Debian Jessie to install Docker. Even the Docker site says this: Docker on Debian Jessie. But that command does not work as of this writing.
- Debian doesn't seem to have an official base image, just a semi-official base image, but really not really, based upon some reading which says that some kluges went into building the base image.
- Some other pointers for a Debian Jessie base image stuff: debian docker base image, Installing Docker on Debian Jessie (from 2013), Installing Docker in Debian (testing/Jessie) (also 2013) , CloudCreateDockerImage
But then, all this might be irrelevant. There is something called CoreOS, which is a ground up solution for running containers.
Wednesday, December 3. 2014
VMware Virtual CPU States
- vCPU: one or more CPUs assigned to a virtual machine
- pCPU: one or more physical CPUs in a host, with a physical slot containing one or more physical cores
- Run: actually performing work. Compare load of one vCPU with others to ensure load is balanced. If all are heavily used, then may additional vCPU resources.
- Wait: two sub-states: idle-wait - nothing to do, io-wait - waiting for to read/write storage. A high io-wait may indicate performance issues on the underlying storage
- Ready: the vCPU is ready to work, but a pCPU has not yet been allocated. <10% may be good. >20% may be bad as allocation process is not being efficient or process resources are scarce. Check for CPU affinity if things seem unbalanced across pCPUs.
Sunday, November 16. 2014
Unikernels - Light Weight Linux
I may not be making use of the Unikernel in the correct context, but what came to mind is the ability to take a Linux distribution and trim it right down to the bone. Some distributions make this easy:
- NixOSK: is a Linux distribution with a unique approach to package and configuration management. Built on top of the Nix package manager, it is completely declarative, makes upgrading systems reliable, and has many other advantages.
- T2 System Development Environment: is a flexible Open Source System Development Environment or Distribution Build Kit (others might even name it Meta Distribution). T2 allows the creation of custom distributions with state of the art technology, up-to-date packages and integrated support for cross compilation.
- Linux From Scratch: provides you with step-by-step instructions for building your own custom Linux system, entirely from source code.
- boot2docker: is a lightweight Linux distribution based on Tiny Core Linux made specifically to run Docker containers. It runs completely from RAM, weighs ~27MB and boots in ~5s (YMMV).
Unikernels and Immutable Infrastructure
I'm working in an environment where we need to manage drive space, memory utilization, and processor efficiency. Just because computer gear seems inexpensive, when looked at scale, all those pennies add up. I am starting to encounter more and more discussions on what other's do for keeping their systems in check.
There are a number of ideas which, when linked together, produce a sum greater than the parts: Immutable Infrastructure and Unikernels.
For the company I work for, our IaaS management systems consist of a good number of virtual guests running specialized, individualized functions. Something akin to the adage of 'one service per server'. Even though security and flexibility contribute to the list of 'strong points', it has certain drawbacks: every server has the full operating system and all the associated trappings: logging, file systems, and many many unused packages, which are loaded by default.
Along comes docker, which makes use of the LXC approach one kernel, but protected run time environments, which reduces the footprint somewhat. But there still is the high over head of unused packages coming along for the ride.
Unikernels provide a DevOps pattern of getting your application as close to the metal as possible and reducing the overhead of unnecessary packages and drivers. This makes more efficient use of drive space, memory space, and processor utilization. Some call this design pattern Cloud Operating Systems.
Once you have these lightweight 'functional objects' available, you can start them and stop them as needed. If you need them changed, you don't change something in the object, you simply create a new one and replace the old one. This actually is extremely beneficial. Rather than undoing a change gone wrong, you roll back a change to the previous version of an object. This provides an inherent degree of versioning in a production environment. And allows for gradual roll-outs, rather than fork-lift style rebuilds.
Unikernels and Immutable Infrastructure written by Darren Rush goes into Unikernels and Immutable Intrastructure in more depth. At the end of the article, more links to more related columns. One of which lands at Florian Motlik's article of Why you should build an Immutable Infrastructure, which expands on those same concepts, provides more links, one of which shines the spotlight on a different definition of the same subject: Disposable Components.
It is easy to get side tracked into further refinements of the same subject, namely Network Function Virtualization. At a past Usenix conference, there was a talk on ClickOS and the Art of Network Function Virtualization, a Unikernel of sorts, based upon Xen and a custom network path, which makes 'network middleboxes' more efficient and flexible. Network middle boxes being things like firewalls, intrusion detection systems, shaping, inspection, monitoring, .... all running in various series/parallel designs.
Sunday, July 27. 2014
Datacenter Network Designs
I have seen a variety of datacenter designs. Some are layer 2 based, some lean more towards layer 3 based. To see what the 'big guys' are doing, I've been watching the release of some recent Internet Drafts.
One of them is from Microsoft/Arista/Facebook employees, and titled: Use of BGP for routing in large-scale data centers. draft-lapukhov-bgp-routing-large-dc-07.
It discusses using a Clos Network physical architecture with EBGP as the routing protocol, without any IGP protocol. It is said able to scale up to thousands and thousands of servers in a redundant, load balancing fashion. This type of network is good for captive, single tenant, style networks.
Another document references Framework for DC Network Virtualization. This document describes some vendor neutral common terminology and concepts. It also provides the ground work for talking about the layer-2/layer-3 design of the underlying network, the mechanism of load balancing multiple tenants over a common hypervisor group, and how to deal with any particular tenant when spread over multiple datacenters.
Another document, VXLAN: A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks goes into some specifics of an overlay network. 2018/08/26: this has been released as rfc 7348.
Saturday, November 23. 2013
libvirt_lxc has high (100%) CPU
On my Debian 7 Wheezy systems running LXC, I have recently seen libvirt_lxc go up to 100% cpu utilization on a continuous basis. The brute force fix was to restart the guest. But that isn't a good idea.
There is a suggestion at [Bug 1005570] libvirt_lxc consumes 100% cpu - msg#00023 that:
When this 100% cpu problem occurs, can anyone tell me if they still see a process running as a child of the 'libvirt_lxc' process ? Whether a zombie or normal process.
Looking at the code, the only way I can see us getting into this problem, is if the guest closed its console (STDIO) streams, but did not exit. We should always get SIG_CHILD if the process exited, so I'm thinking perhaps the container's init process closed stdin/out/err, but kept running
When I logged in via the lxc console as root, the high cpu utilization went a way.
As a side note, to login as root via the lxc console, I had to add 'pts/0' to /etc/securetty. I also only have one tty listed in /etc/inittab:
# Normally not reached, but fallthrough in case of emergency. z6:6:respawn:/sbin/sulogin 1:2345:respawn:/sbin/getty 38400 tty1 #2:23:respawn:/sbin/getty 38400 tty2 #3:23:respawn:/sbin/getty 38400 tty3 #4:23:respawn:/sbin/getty 38400 tty4 #5:23:respawn:/sbin/getty 38400 tty5 #6:23:respawn:/sbin/getty 38400 tty6
Let me see if these settings actually help prevent further recurrence of the high cpu usage in libvirt_lxc.
Monday, August 12. 2013
Linux Containers (LXC) on Debian Wheezy 7.1 With OpenVSwitch
Now that the networking side of things are complete with Quagga and OpenVSwitch, I can start working further up the technology layers. The next step is to work on virtualization technologies and their management. The focus of this article is to get Linux Containers (LXC) working within a Debian Wheezy 7.1 environment. Continue reading "Linux Containers (LXC) on Debian Wheezy 7.1..." »
Sunday, August 11. 2013
Understanding OpenVSwitch Configuration
My previous openvswitch article dealt with building a recent release on one system, migrating the .deb files to another system, installing them, and getting the daemon to run upon reboot. Last year I also wrote about some useful configuration commands, which dealt with simple access port style interfaces. Continue reading "Understanding OpenVSwitch Configuration" »


