Build PXEBoot/DnsMasq Server
For this example, I will be using the 'Official Release' netinst Debian Installer Stretch Alpha 4 Release found at https://www.debian.org/devel/debian-installer/ .
I typically build servers with a minimal install. For the boot server, I use two interfaces:
- an outside interface to the internet for external base services, which are 'proxied' for use to the internal servers
- an inside interface for serving the servers I build. These servers are unable to see (are not routed) to the outside world directly, which provides some measure of security.
I will be using the ISO labelled as debian-stretch-DI-alpha4-amd64-netinst.iso (64 bit net based install). These examples use VirtualBox for the purposes of demonstration.
I have the VirtualBox extension pack installed.
For the pxeboot server only, the VirtualBox configuration is:
- a 16G storage device labelled as sda
- two network adapters, the first bridged to the outside world, the second attached to an internal host-only network. Each network is configured with the Intel PRO/1000 MT Server (82545EM) network card.
- audio can be disabled
- iso attached to the IDE controller
- boot order with harddrive first, cd second
For this example, the second interface is associated with the host-only network. The host-only network has been configured with out dhcp, with the host (the computer running VirtualBox) having an address of 192.168.56.1
I use expert mode to build the server. There are variations on the theme, but these are the settings I've selected in the menus for this configuration:
- advanced options
- expert install
- choose language: English
- country: other, Caribbean, Bermuda
- localle settings: default of United States
- no additional locales
- keyboard: default of American English
- cd: should auto-detect and mount
- installer components: no additional components needed
- network hardware: auto-detection
- configure network: use enp0s17 as the first interface
- auto-configure network: yes if dhcp is available, no if static configuration required
- hostname: dmsq01 (or one matching your own requirements)
- domain name: example.com (or one matching your own requirements)
- users and passwords, enable shadow passwords
- login as root: yes (this allows login as root, substitute your own requirements)
- supply a password
- a normal user account is not created (create one if your policies require it)
- configure the clock: NTP: no (allow if you have an ntp server available)
- time zone: Atantic/Bermuda
- detect disks: auto-detection
- partition disks: guided - use entire disk
- partition: select VBOX HARDDISK
- partitioning scheme: all in one partition (change if your policies require it)
- finish partitioning and write changes
- install the base system
- kernel to install: linux-image-4.2.0-1-amd64 (as of 2015/12/29)
- drivers to include: targeted
- configure package manager:
- network mirror: yes
- protocol: http (change as policies require)
- mirror country: US, ftp.us.debian.org (or one nearer to you)
- use non-free software: default of no
- use contrib software: default of no
- services to use: defaults: security updates only
- select and install software:
- popularity contest: no
- software to install: deselect everything, select 'SSH server' only
- install grub on hard disk
- install grub on master boot record
- boot loader installation: /dev/sda
- EFI removable path: default no
- finish the installation
- system clock to UTC: default yes
- installation complete: continue
- reboots to login prompt
Within the VirtualBox console of the machine, login and determine the ip address with the command 'ip addr'. In addition,
run the following commands to allow root login via ssh:
$ sed -i 's/^PermitRootLogin .*/PermitRootLogin yes/' /etc/ssh/sshd_config
$ systemctl restart sshd
Since the console doesn't allow cut and paste, it is easier to ssh into the guest, and perform the remaining activities
via an ssh shell.
Build DnsMasq server for controlling PXEBoot Process
Once logged in via the ssh shell, this next section installs and configures the specific packages necessary for converting the installed server
into a pxeboot server. The following primary packages are installed:
- apt-cacher-ng: acts as a package proxy and caching mechanism so that new guests don't have to access an outside package server
- dnsmasq: provides pxeboot, dhcp, tcp, and dns services for guest servers
Install the basic packages:
$ apt-get install apt-cacher-ng dnsmasq less python
Confirm that there is a second interface called enp0s8 on the server:
$ ip link
1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp0s8: mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether 08:00:27:33:43:fa brd ff:ff:ff:ff:ff:ff
3: enp0s17: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
link/ether 08:00:27:1e:d2:20 brd ff:ff:ff:ff:ff:ff
Interface enp0s8 we will designate as the internal interface, and is used for management of additional guests.
The basic defaults that dnsmasq uses are represented in /etc/dnsmasq.conf. To over-ride those values, we'll create a file called
/etc/dnsmasq.d/local and supply the following content.
interface=enp0s8
dhcp-fqdn
domain=example.com
dhcp-range=192.168.56.131,192.168.56.149,255.255.255.0,1h
dhcp-boot=pxelinux.0,pxeserver,192.168.56.10
pxe-service=x86PC, "Install Debian Stretch",pxelinux,192.168.56.10
enable-tftp=enp0s8
tftp-root=/var/local/tftp,enp0s8
dhcp-client-update
dhcp-fqdn
log-queries=extra
server=8.8.8.8
# find by dnsmasq --help dhcp
dhcp-option=tag:gwon56,option:router,192.168.56.10
dhcp-option=option:ntp-server,192.168.56.10
Append the following to /etc/network/interfaces:
allow-hotplug enp0s8
iface enp0s8 inet static
address 192.168.56.10
netmask 255.255.255.0
Activate the address on the interface and bring it up:
$ ip addr add 192.168.56.10/24 dev enp0s8
$ ip link set up dev enp0s8
Confirm the address has been applied, and that the interface is up:
$ ip link
$ ip addr
Install the pxeboot tools:
$ mkdir /var/local/tftp
$ pushd /var/local/tftp
## fyi: installs the netboot which matches the cd we used, but we'll use a more current installer, so skip this
## wget http://ftp.debian.org/debian/dists/testing/main/installer-amd64/current/images/netboot/netboot.tar.gz
## this is the more up-to-date installer we'll use:
$ wget http://d-i.debian.org/daily-images/amd64/daily/netboot/netboot.tar.gz
## if there is a certificate problem (which may occur during kernel transitions), use this command:
## wget --no-hsts --server-response --no-check-certificate http://d-i.debian.org/daily-images/amd64/daily/netboot/netboot.tar.gz
$ tar zxvf netboot.tar.gz
$ mkdir seeds
$ popd
# following directory is for our company specific configuration files
$ mkdir /etc/qvsl
$ cd /etc/qvsl
Restart the dnsmasq service:
$ systemctl restart dnsmasq
With the pxeboot server in place, another guest can be built in order to obtain an appropriate preseed file.
Build a server for obtaining the raw seed file
In VirtualBox, another guest can be created with the following settings:
- a single 8G storage device labelled as sda
- one network adapter, connected to the host-only network, and configured with the Intel PRO/1000 MT Server (82545EM) network card.
- audio can be disabled
- no iso is attached
- boot order with harddrive first, network second (cd and floppy are disabled)
By starting this guest, it should boot off the pxeboot server (cancel when asked for a startup disk):
- advanced options
- expert install
- choose language: English
- country: other, Caribbean, Bermuda
- locale settings: default of United States
- no additional locales
- keyboard: default of American English
- network hardware: auto-detection
- auto-configure network: yes (dnsmasq will supply the address)
- hostname: seed
- domain name: example.com (or one matching your own requirements)
- choose a mirror:
- protocol for downloads: http
- archive mirror: enter manually, 192.168.56.10:3142
- mirror directory: default as /debian/
- no http proxy
- version to install: stretch - testing
- download installer components:
- installer components: continue with defaults
- setup users and passwords:
- enable shaddow passwords
- allow login as root: yes
- supply root password
- create normal user account: no
- configure the clock: ntp: no
- timezone: Atlantic/Bermuda
- detect disks:
- partition disks: guided - use entire disk
- select sda, VBOX HARDDISK
- all files in one partition
- finish partitioning and write changes: yes
- install base system:
- kernel to install: linux-image-4.3.0-1-amd64
- drivers to include: targeted
- use non-free software: default no
- use contrib software: default no
- enable source repositories: no
- services to use: un-select all including security updates
- select and install software
- participate in package survey: no
- software selection: deselect everything, select 'SSH server' only
- install grub on harddrive
- install grub to master boot record: yes
- boot loader installation: /dev/sda
- grub to EFI removable media path: default no
- finish the installation
- system clock to UTC: default yes
- installation complete: continue
- reboots to login prompt
From the VirtualBox console, login to the newly built server, and we'll get the base content of the seed file:
$ apt-get install debconf-utils
$ debconf-get-selections --installer > seed.txt
$ debconf-get-selections >> seed.txt
$ scp seed.txt root@192.168.56.10:/etc/qvsl/
# we no longer need this machine, so shut it down, then delete it
$ shutdown -h now
Fill in the details for the preseed / auto-build process
Back on the dmsq01 server, the file seed.txt should be in /etc/qvsl. One line needs to be mangled, otherwise all packages get
installed rather than just a select few:
$ sed -i 's/.*SSH server$/# SSH server/' seed.txt
In the local directory, create a python script file with the name /etc/qvsl/seed.sort.py and supply the following content:
#
# author: Raymond Burkholder
# rburkholder@quovadis.bm
# created: 2015/08/10
#
# sorts and collates debconf output:
# debconf-get-selections --installer > seed.txt
# debconf-get-selections >> seed.txt
# python seed.sort.py < input.seed
#
# NOTE: in the seed file, the following line must be commented out,
# otherwise many packages will be installed, rather than just the chosen ones
# d-i tasksel/first multiselect SSH server
import sys
import re
import os
# some output modifiers, which could be done from command line at some point
di_only = True # True: exclude other entries d-i entry available, False to print all entries
answers_only = False # True: only print entries with answers
def buildfields( s ):
fields = ()
match = re.match( '^(\S+)\s+(\S+)\s+(\S+)\s+(\S+)', s )
if ( None != match ):
fields = ( match.group(1), match.group(2), match.group(3), match.group(4) )
else:
match = re.match( '^(\S+)\s+(\S+)\s+(\S+)', s )
if ( None == match ):
print 'broken 1'
os._exit(0)
else:
fields = ( match.group(1), match.group(2), match.group(3), '' )
if ( 4 != len(fields) ):
print 'broken 2'
os._exit(0)
return fields
def buildkey( fields ):
key = fields[0] + ' ' + fields[1] + ' ' + fields[2]
return key
# http://stackoverflow.com/a/17927520/2730971
f = sys.stdin
if len(sys.argv) > 1:
f = open(sys.argv[1])
# state:
# 0 nothing found yet
# 1 found #, on transition in, save/clear, append
# 2 [1-zA-Z]: save/clear, then append
# 3 empty or white space: append
# 4 anything else: error, or ignore
state = 0 # state transitions
buf = "" # accumulation of statements
dict = {} # check for duplicate keys at some point, may therefore need a func
key = ""
fields = ()
# parse the file and build up dictionary with entries and associated comments
for line in f:
if ( 0 < len(line) ):
text = line[:-1]
if ( 0 < len(text) ):
if ( '#' == text[0] ):
if ( 1 != state ): # save what came before
if ( 0 < len(key) ):
if ( not ( key in dict ) ): # ignore duplicaetes further in file
#print 'added dict 1'
dict[key] = ( fields, buf )
key = ""
buf = ""
fields = ()
buf += line
state = 1 # state: comment
else:
if ( ( 'a' <= text[0] and 'z' >= text[0] ) or ( 'A' <= text[0] and 'Z' >= text[0] ) ):
if ( 2 == state ):
if ( not ( key in dict ) ): # ingnore duiplicates further in file
#print 'added dict 2'
dict[key] = ( fields, buf )
key = ""
buf = ""
fields = ()
fields = buildfields( text )
key = buildkey( fields )
buf += line
state = 2 # state: regular character
else:
if ( ' ' == line[0] or '\t' == line[0] ):
buf += line
state = 3 # state: empty line or continuation
else:
state = 4 # state: nothing useful
#print( "state %s" % (state) )
#print line[0]
if ( 0 < len(key) ): # finish any stragglers
if ( not ( key in dict ) ): # ignore duplicaetes further in file
#print 'added dict 3'
dict[key] = ( fields, buf )
# parse out second element of line so can sort the dictionary
# and build a new dictionary with multiple answers of same key
newdict = {}
#print len(keys)
for akey in list(dict.keys()):
#print dict[akey]
#print match.group(2)
# maybe choice with answers here but some answers are legitimate blanks
newkey = dict[akey][0][1] # pull out list, pull out fields, second field in list
if ( newkey in newdict ):
#alist = newdict[newkey]
#alist += [ dict[akey] ]
newdict[newkey] += [ dict[akey] ]
else:
newdict[newkey] = [ dict[akey] ]
# print out the sorted list of questions, combining debconf and d-i
keys = list( newdict.keys() )
keys.sort()
for akey in keys:
#print akey
alist = newdict[akey]
#print( "%d, %s" % ( len(alist), akey ) )
if ( di_only ):
difound = False
for item in alist:
#match = re.match( '^(\S+)', item[ 1 ][0] )
#if ( None != match ):
if ( 'd-i' == item[0][0] ):
difound = True
print item[1]
if ( not difound ):
for item in alist:
print item[1]
else:
for item in alist:
print item[1]
Place the following into a file called /etc/qvsl/seed.prefix.txt. These are answer overrides specific to our needs.
In this particular instance, a password for root is included, a customized disk partitioning scheme is used,
and the salt-minion package is automatically installed (which allows further automated configuration).
Adjust to your own specific requirements.
# rebuild with:
# python seed.sort.py < input.seed > output.seed
# Enable source repositories in APT?
apt-setup-udeb apt-setup/enable-source-repositories boolean false
# Kernel to install:
# Old Choices: linux-image-4.1.0-1-amd64,linux-image-4.1.0-1-rt-amd64,linux-image-amd64,linux-image-rt-amd64, none
bootstrap-base base-installer/kernel/image select linux-image-4.3.0-1-amd64
# Choose software to install:
# Choices: Debian desktop environment, ... GNOME, ... Xfce, ... KDE, ... Cinnamon, ... MATE, ... LXDE, web server, print server, SSH
tasksel tasksel/first multiselect ssh-server
# for internal use; can be preseeded
d-i partman-auto/method string regular
# Select disk to partition:
# Choices:
d-i partman-auto/select_disk select /dev/sda
# Do you want to return to the partitioning menu?
partman-basicfilesystems partman-basicfilesystems/no_swap boolean false
# Waiting time (in seconds) for link detection:
d-i netcfg/link_wait_timeout string 2
# for internal use; can be preseeded
#d-i base-installer/kernel/linux/extra-packages string salt-minion
bootstrap-base base-installer/kernel/linux/extra-packages string salt-minion
## **
# Continue with the installation?
partman-base partman/confirm_nochanges boolean false
## **
# for internal use; can be preseeded
partman-base partman/default_filesystem string ext4
## **
# Write the changes to disks?
partman-base partman/confirm boolean true
## **
# Write the changes to disks?
partman-base partman/confirm_nooverwrite boolean false
## **
# for internal use; can be preseeded - for creating complicated disk configurations
partman-base partman/early_command string
## **
# Partitioning scheme:
# Choices:
#partman-auto partman-auto/choose_recipe select
d-i partman-auto/choose_recipe select custom1
## **
# for internal use; can be preseeded
d-i partman-auto/expert_recipe string \
custom1 :: \
75 1000 100 ext4 \
$primary{ } $bootable{ } \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /boot } \
. \
1000 4000 5000 ext4 \
$primary{ } \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ / } \
. \
5000 10000 1000000000 btrfs \
$primary{ } \
method{ format } format{ } \
use_filesystem{ } filesystem{ btrfs } \
mountpoint{ /var } \
. \
4000 8000 10000 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /home } \
. \
1000 1000 4000 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /tmp } \
.
# options/ro{ ro }
# 64 512 300% linux-swap
# method{ swap } format{ }
## **
# Partitioning scheme:
# Choices: All files in one partition (recommended for new users), Separate /home partition, Separate /home\, /var\, and /tmp partit$
#partman-auto partman-auto/choose_recipe select /lib/partman/recipes/30atomic
# You can choose one of the three predefined partitioning recipes:
# - atomic: all files in one partition
# - home: separate /home partition
# - multi: separate /home, /usr, /var, and /tmp partitions
#d-i partman-auto/choose_recipe select atomic
# This makes partman automatically partition without confirmation, provided
# that you told it what to do using one of the methods above.
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean false
d-i partman/confirm_nooverwrite boolean true
## **
## Controlling how partitions are mounted
# The default is to mount by UUID, but you can also choose "traditional" to
# use traditional device names, or "label" to try filesystem labels before
# falling back to UUIDs.
#d-i partman/mount_style select uuid
#d-i partman/mount_style select traditional
## **
# for internal use; can be preseeded
d-i partman-auto/disk string /dev/sda
## **
# for internal use; can be preseeded
base-installer base-installer/install-recommends boolean false
d-i base-installer/install-recommends boolean false
## **
# Allow login as root?
user-setup-udeb passwd/root-login boolean true
## **
# Disable SSH password authentication for root?
d-i openssh-server/permit-root-login boolean true
## **
# for internal use; can be preseeded
d-i debian-installer/exit/poweroff boolean false
## **
# for internal use; can be preseeded
d-i debian-installer/exit/halt boolean false
## **
# for internal use only
d-i debian-installer/exit/always_halt boolean false
## **
# Installation complete
finish-install finish-install/reboot_in_progress note
# Root password, either in clear text
d-i passwd/root-password password PreSeed0
d-i passwd/root-password-again password PreSeed0
## **
# GRUB install devices:
# Choices: /dev/sda (8589 MB; VBOX_HARDDISK), - /dev/sda1 (8185 MB; /)
#grub-pc grub-pc/install_devices multiselect /dev/disk/by-id/ata-VBOX_HARDDISK_VBdeb9f519-8a3c917e
# Device for boot loader installation:
# Choices: Enter device manually, /dev/sda (ata-VBOX_HARDDISK_VBdeb9f519-8a3c917e)
d-i grub-installer/choose_bootdev select /dev/sda
## **
# for internal use; can be preseeded (deprecated)
d-i netcfg/disable_dhcp boolean true
## **
# Is this information correct?
d-i netcfg/confirm_static boolean true
Build a composite file where our answers override the default answers:
$ cp seed.prefix.txt qvsl.seed
$ cat seed.txt >> qvsl.seed
$ python seed.sort.py < qvsl.seed > qvsl.seed.pxe
Now we can prepare to auto-build a machine.
In VirtualBox, create a new machine, call it 'test01' with the following attributes:
- a 16G storage device labelled as sda
- one network adapter connected to the internal host-only network, and configured with the Intel PRO/1000 MT Server (82545EM) network card.
- audio can be disabled
- do not attach an iso
- boot order with harddrive first, network second, no cd or floppy
Make note of the supplied mac address from the network card, which in this case, is: 080027CD57B7
Substituting and formatting your mac address as shown (lower case), append a line to the bottom of /etc/dnsmasq.d/local:
dhcp-host=08:00:27:cd:57:B7,set:gwon56,192.168.56.20,test01,300
This line supplies gateway of 192.168.56.10,
supplies an address to the machine of 192.168.56.20, and sets dhcp time out to 5 minutes (300 seconds)
Dhcp will also supply the domain name.
Restart the service:
$ systemctl restart dnsmasq
Place the following into /etc/qvsl/boot.pxe, this is indicates to the pxeboot process where to find its various parameters:
D-I config version 2.0
default debian-installer/amd64/boot-screens/vesamenu.c32
DEFAULT auto
SAY Booting new build ....
LABEL auto
menu label ^Auto
kernel debian-installer/amd64/linux
append vga=788 initrd=debian-installer/amd64/initrd.gz auto=true url=tftp://192.168.56.10/seeds/qvsl.seed.pxe domain=rpb.qvs.bm hostname=undefined interface=enp0s17
# append DEBCONF_DEBUG=5 to the line for debugging
prompt 0
timeout 1
Make the following links into the tftp directory (use your mac address, sutbstituting hypens for colons, and prepending '01-'):
$ ln -s /etc/qvsl/qvsl.seed.pxe /var/local/tftp/seeds/qvsl.seed.pxe
$ ln -s /etc/qvsl/boot.pxe /var/local/tftp/pxelinux.cfg/01-08-00-27-cd-57-b7
By making the mac link this way, multiple machines can use the same boot.pxe file.
By running the following command, you can watch the boot process and troubleshoot accordingly:
$ tail -f /var/log/daemon.log
Start the machine to be auto-built
Now start the VirtualBox machine 'test01' created earlier, press cancel when it requests a start up disk.
It should now auto-boot and auto-build.
Once it automatically restarts, you should be able to login as root at the console using the password as located in the preseed file.