WARNING: This page is archived and not updated. Please bear in mind any information may be incorrect and any code may not work. Use at your own risk.
James Gardner: Home > Blog > 2009 > An Introduction to KVM and JeOS...

An Introduction to KVM and JeOS Images on Ubuntu Intrepid

Posted:2009-03-10 02:01
Tags:Web, Desktop Software, Virtulization, Hosting

This started off as a few notes about KVM and making a JeOS but has rapidly turned into a huge article. It is still growing and I haven't even read it through yet so please treat as draft. It is published here as much so I don't lose it as for any other reason!

A few recent announcements have left the future of Xen slightly uncertain. In 2008 the Ubuntu Server team that they would focus on KVM rather than Xen (CNET article and the actual statement) and Red Hat made similar statements in February this year (blogged here). I'm sure Xen isn't going away any time soon but if KVM is flavour of the month it is probably worth a look.

The Ubuntu project documents how to build and run KVM virtual machines in the following documents which form part of the 8.10 Intrepid release:

and in this article on the wiki:

I've followed these articles and to be honest didn't find them very useful either because they were incomplete or slightly misleading so although they are essential reading, I'd recommend reading them after the introduction below.

Better documentation can be found in these articles:

Introduction

Although Xen and KVM are different technologies, they are very similar from a user's point of view. With both systems the tricky parts are building the image in the first place and configuring the networking. Intrepid adds another ingredient to the mix which is libvert and the virsh shell. These are tools which allow you to manage a range of virtual machines through the same interfaces, regardless of the underlying virtualisation technology. To confuse matters further, KVM images are actually loaded with Qemu and rather than specifying the config file directly, you specify its name and the configuration itself is loaded from an XML file in /etc/vmbuilder/qemu/. Also, you can't easily connect to a serial console from the command line. Instead you use the virt-manager GUI tool. All this will become clear as you follow this article.

Action Xen/Etch KVM/Intrepid
Virtual machine creation xen-create-image vm-builder
Start VM xm create /path/to/vm1.cfg virsh -c qemu:///system start vm1
Stop VM xm shutdown 1 virsh -c qemu:///system stop vm1
Connect to console xm console 1 Use the GUI

Table 1: Comparison of Xen commands on Debian Etch with KVM commands on Ubuntu Intrepid

Let's Get Started

To see whether your CPU supports virtualization run this command:

egrep '(vmx|svm)' /proc/cpuinfo

Since I've got the VMX extensions I get a few lines of output. I can therefore install libvirt and kvm:

sudo apt-get install kvm libvirt-bin

After installing libvirt-bin, the user used to manage virtual machines will need to be added to the libvirtd group. Doing so will grant the user access to the advanced networking options.

sudo adduser $USERNAME libvirtd

You'll need to log in and log out for the changes to take effect.

To make life easier you can install the Virtual Machine Manager, a GTK application for managing virtual machines with libvirt as well as for connecting to running machines (it uses VNC internally):

sudo apt-get install virt-manager

You can start it like this:

virt-manager -c qemu:///system

You can even connect to a libvirt service running on another host by entering the following in a terminal prompt:

virt-manager -c qemu+ssh://virtnode1.mydomain.com/system

Once connected you'll see something like this:

../2009/03/virt-manager1.thumbnail.png

There is also a standalone view you can install called virt-viewer but you don't really need it because virt-manager already provides similar functionality.

Creating a Virtual Machine

Ubuntu Intrepid comes with a tool called python-vm-builder for creating virtual machine images. It is a newer version of the ubuntu-vm-builder which came with Hardy. Install it like this:

sudo apt-get install python-vm-builder

Base Parameters

As this example is based on KVM and Ubuntu 8.10 (Intrepid Ibex), and we are likely to rebuild the same virtual machine multiple time, we'll invoke vmbuilder with the following first parameters:

sudo vmbuilder kvm ubuntu --suite intrepid --flavour virtual --arch i386  -o --libvirt qemu:///system

The --suite defines the Ubuntu release, the --flavour specifies that we want to use the virtual kernel (that's the one used to build a JeOS image), the --arch tells that we want to use a 32 bit machine, the -o tells vmbuilder to overwrite the previous version of the VM (although this often doesn't work) and the --libvirt tells to inform the local virtualization environment to add the resulting VM to the list of available machines.

Notes:

  • Because of the nature of operations performed by vmbuilder, it needs to have root privilege, hence the use of sudo.
  • If your virtual machine needs to use more than 3Gb of ram, you should build a 64 bit machine (--arch amd64).
  • Until Ubuntu 8.10, the virtual kernel was only built for 32 bit architecture, so if you want to define an amd64 machine on Hardy, you should use --flavour server instead.

Before you run the command above you should install Apt Proxy.

Installing Apt Proxy

If you are creating lot's of VMs it can be costly to keep downloading the packages you need. The Ubuntu JeOS guide recommends apt-mirror but that downloads all the packages in advance which isn't what we want either. Instead we'll use apt-proxy (https://help.ubuntu.com/community/AptProxy). It is possible to configure apt-proxy to handle your usual system updates too this is a bit more complex and we're not aiming for perfection here, just to reduce download times as much as possible.

sudo apt-get install apt-proxy

With apt proxy installed let's create an image, using the --mirror http://localhost:9999/ubuntu option to ensure the packages are cached so they can be re-used next time.

sudo vmbuilder kvm ubuntu --suite intrepid --flavour virtual --arch i386  -o --libvirt qemu:///system --mirror http://localhost:9999/ubuntu

The command takes about 10 minutes to complete. Here's the output:

2009-03-09 19:28:06,138 INFO     Creating disk image: /tmp/vmbuilderPbLfCO/disk0.img
2009-03-09 19:28:06,143 INFO     Adding partition table to disk image: /tmp/vmbuilderPbLfCO/disk0.img
2009-03-09 19:28:06,160 INFO     Adding type 1 partition to disk image: /tmp/vmbuilderPbLfCO/disk0.img
2009-03-09 19:28:06,167 INFO     Adding type 3 partition to disk image: /tmp/vmbuilderPbLfCO/disk0.img
2009-03-09 19:28:06,175 INFO     Creating loop devices corresponding to the created partitions
2009-03-09 19:28:06,198 INFO     Creating file systems
2009-03-09 19:28:06,208 INFO     mke2fs 1.41.3 (12-Oct-2008)
2009-03-09 19:28:06,959 INFO     Mounting target filesystems
2009-03-09 19:28:06,967 INFO     Installing guest operating system. This might take some time...
2009-03-09 19:29:35,601 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:29:35,602 INFO     Moving old data out of the way
2009-03-09 19:29:35,602 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:29:35,812 INFO     Searching for GRUB installation directory ... found: /boot/grub
2009-03-09 19:29:36,626 INFO     Searching for default file ... Generating /boot/grub/default file and setting the default boot entry to 0
2009-03-09 19:29:36,630 INFO     Searching for GRUB installation directory ... found: /boot/grub
2009-03-09 19:29:36,648 INFO     Testing for an existing GRUB menu.lst file ...
2009-03-09 19:29:36,648 INFO
2009-03-09 19:29:36,649 INFO     Could not find /boot/grub/menu.lst file.
2009-03-09 19:29:36,652 INFO     Generating /boot/grub/menu.lst
2009-03-09 19:29:36,784 INFO     Searching for splash image ... none found, skipping ...
2009-03-09 19:29:36,947 INFO     grep: /boot/config*: No such file or directory
2009-03-09 19:29:37,069 INFO     Updating /boot/grub/menu.lst ... done
2009-03-09 19:29:37,069 INFO
2009-03-09 19:29:37,311 INFO     Searching for GRUB installation directory ... found: /boot/grub
2009-03-09 19:29:37,421 INFO     Searching for default file ... found: /boot/grub/default
2009-03-09 19:29:37,430 INFO     Testing for an existing GRUB menu.lst file ... found: /boot/grub/menu.lst
2009-03-09 19:29:37,570 INFO     Searching for splash image ... none found, skipping ...
2009-03-09 19:29:37,633 INFO     grep: /boot/config*: No such file or directory
2009-03-09 19:29:37,749 INFO     Updating /boot/grub/menu.lst ... done
2009-03-09 19:29:37,750 INFO
2009-03-09 19:29:37,813 INFO     Searching for GRUB installation directory ... found: /boot/grub
2009-03-09 19:29:41,028 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:29:41,028 INFO     Done.
2009-03-09 19:29:43,692 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:29:43,692 INFO     Running depmod.
2009-03-09 19:29:43,693 INFO     update-initramfs: Generating /boot/initrd.img-2.6.27-11-server
2009-03-09 19:29:47,923 INFO     Running postinst hook script /usr/sbin/update-grub.
2009-03-09 19:29:48,232 INFO     Searching for GRUB installation directory ... found: /boot/grub
2009-03-09 19:29:48,665 INFO     Searching for default file ... found: /boot/grub/default
2009-03-09 19:29:48,672 INFO     Testing for an existing GRUB menu.lst file ... found: /boot/grub/menu.lst
2009-03-09 19:29:48,795 INFO     Searching for splash image ... none found, skipping ...
2009-03-09 19:29:48,848 INFO     grep: /boot/config*: No such file or directory
2009-03-09 19:29:48,913 INFO     Found kernel: /boot/vmlinuz-2.6.27-11-server
2009-03-09 19:29:49,184 INFO     Replacing config file /var/run/grub/menu.lst with new version
2009-03-09 19:29:49,218 INFO     Updating /boot/grub/menu.lst ... done
2009-03-09 19:29:49,219 INFO
Extracting templates from packages: 100%
2009-03-09 19:29:56,791 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:29:57,217 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:29:57,321 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:29:57,507 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:29:57,621 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:29:57,859 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:29:57,919 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:03,999 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,000 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,000 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,001 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,002 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,002 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,003 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,004 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,004 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,005 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,006 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,006 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,007 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,008 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,008 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,009 INFO     find: `/var/cache/fontconfig': No such file or directory
2009-03-09 19:30:04,009 INFO     find: `/var/cache/fonts': No such file or directory
2009-03-09 19:30:04,010 INFO     find: `/var/cache/anthy': No such file or directory
2009-03-09 19:30:04,011 INFO     find: `/var/lib/belocs': No such file or directory
2009-03-09 19:30:04,012 INFO     find: `/var/lib/gconf': No such file or directory
2009-03-09 19:30:04,013 INFO     find: `/var/lib/defoma': No such file or directory
2009-03-09 19:30:04,013 INFO     find: `/var/log/installer': No such file or directory
2009-03-09 19:30:04,014 INFO     find: `/cdrom': No such file or directory
2009-03-09 19:30:04,014 INFO     find: `/media/cdrom': No such file or directory
2009-03-09 19:30:04,015 INFO     find: `/usr/share/fonts': No such file or directory
2009-03-09 19:30:04,015 INFO     find: `/var/lib/anthy': No such file or directory
2009-03-09 19:30:04,016 INFO     find: `/var/lib/defoma': No such file or directory
2009-03-09 19:30:04,017 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,017 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:04,018 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:06,474 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:06,888 INFO
2009-03-09 19:30:06,889 INFO     Current default timezone: 'Etc/UTC'
2009-03-09 19:30:06,899 INFO     Local time is now:      Mon Mar  9 19:30:06 UTC 2009.
2009-03-09 19:30:06,900 INFO     Universal Time is now:  Mon Mar  9 19:30:06 UTC 2009.
2009-03-09 19:30:06,901 INFO     Run 'dpkg-reconfigure tzdata' if you wish to change it.
2009-03-09 19:30:06,901 INFO
2009-03-09 19:30:06,998 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:11,823 INFO     Can not write log, openpty() failed (/dev/pts not mounted?)
2009-03-09 19:30:17,727 INFO     Updating certificates in /etc/ssl/certs....done.
2009-03-09 19:30:17,727 INFO     Running hooks in /etc/ca-certificates/update.d....done.
2009-03-09 19:30:20,007 INFO     grep: /proc/self/status: No such file or directory
2009-03-09 19:30:20,199 INFO     Use of uninitialized value $x in scalar assignment at /usr/share/perl/5.10/utf8_heavy.pl line 242.
2009-03-09 19:30:20,201 INFO     Use of uninitialized value $x in pattern match (m//) at /usr/share/perl/5.10/utf8_heavy.pl line 243.
2009-03-09 19:30:20,202 INFO     Use of uninitialized value $uni in pattern match (m//) at /usr/bin/ckbcomp line 3109.
2009-03-09 19:30:20,204 INFO     Use of uninitialized value $uni in pattern match (m//) at /usr/bin/ckbcomp line 3109.
2009-03-09 19:30:20,741 INFO     Use of uninitialized value $uni in pattern match (m//) at /usr/bin/ckbcomp line 3109.
2009-03-09 19:30:20,743 INFO     Use of uninitialized value $uni in pattern match (m//) at /usr/bin/ckbcomp line 3109.
2009-03-09 19:30:20,982 INFO     update-initramfs: deferring update (trigger activated)
2009-03-09 19:30:26,856 INFO     Copying to disk images
2009-03-09 19:30:53,699 INFO     Installing bootloader
2009-03-09 19:30:55,956 INFO     Unmounting target filesystem
2009-03-09 19:30:56,460 INFO     Converting /tmp/vmbuilderPbLfCO/disk0.img to qcow2, format ubuntu-kvm/disk0.qcow2
2009-03-09 19:31:42,132 INFO     Cleaning up

The command creates a folder called ubuntu-kvm containing the image created.

$ ls -lah ubuntu-kvm/
total 317M
drwxr-xr-x 2 james james 4.0K 2009-03-09 19:40 .
drwxr-xr-x 3 james james 4.0K 2009-03-09 19:37 ..
-rw-r--r-- 1 james james 317M 2009-03-09 19:40 disk0.qcow2

Because no name was specified for this image the default of ubuntu is used. This means the following config file is created in /etc/libvirt/qemu/ubuntu.xml:

<domain type='kvm'>
  <name>ubuntu</name>
  <uuid>732ca617-ec05-df32-d43c-cb6b18f6d9d8</uuid>
  <memory>131072</memory>
  <currentMemory>131072</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/kvm</emulator>
    <disk type='file' device='disk'>
      <source file='/home/james/vms/tes/ubuntu-kvm/disk0.qcow2'/>
      <target dev='hda' bus='ide'/>
    </disk>
    <interface type='network'>
      <mac address='52:54:00:f2:93:7c'/>
      <source network='default'/>
    </interface>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' listen='127.0.0.1'/>
  </devices>
</domain>

It is this configuration which will be used by libvert when you start the virtual machine.

Changing the Default XML Template

Sometimes you might want a slightly different XML config file to be created. In order to customise how the file is generated you need to copy the template directory structure from /etc/vmbuilder/libvirt/* to the current working directory. The vmbuilder program will then use the customised version instead. Here are the commands:

mkdir -p VMBuilder/plugins/libvirt/templates
cp /etc/vmbuilder/libvirt/* VMBuilder/plugins/libvirt/templates/

One instance where you will need to do this is when you customise the type of networking setup you are using.

Changing an XML File Once it is Created

If you want to customize an XML file after it has been created you need to tell libvirt about the changes by loading the virsh shell and re-defining it:

$ sudo virsh -c qemu:///system
Connecting to uri: qemu:///system
Welcome to virsh, the virtualisation interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # define /etc/libvirt/qemu/ubuntu.xml
Domain ubuntu defined from /etc/libvirt/qemu/ubuntu.xml

Re-Creating a Virtual Machine

If you run the command again you'll see that the existing image is overwritten and that this time there is no network access because all the packages are retrieved from apt proxy:

2009-03-09 19:37:24,271 INFO     ubuntu-kvm exists, and --overwrite specified. Removing..
...

Testing the Virtual Machine

To test the virtual machine you've just created you can use the virsh shell which uses libvert to provide commands to handle virtual machines using a variety of technologies including Xen and KVM, with the same set of commands.

Connect to virsh like this:

virsh -c qemu:///system

Then start the server with this command:

start ubuntu

You won't see much because the basic system doesn't have a console and SSH isn't installed so you can't connect to it. Quit the virsh shell with the machine still running:

quit

Now load the Virtual Machine Manager:

virt-manager

Choose the QEMU hypervisor and the local machine then and open the running virtual machine:

../2009/03/ubuntu-running.thumbnail.png

You can now shutdown the machine either by clicking shutdown on the toolbar or typing shutdown ubnutu at a visrh shell.

A Real Example

Now you've been through the cycle of creating booting and shutting down a KVM image, let's look at a real example. Imagine you want to create a JeOS for running the SquirrelMail web mail program. The image we created would need to be on the network and it would be a good idea for it to run SSH. It would need a user account to allow remote login and it would obviously need squirrelmail installed which in turn requires Apache and PHP.

First let's see the options available to us:

$ vmbuilder -h
Usage: vmbuilder hypervisor distro [options]

Arguments:
  hypervisor            Hypervisor. Valid options: xen kvm vmw6 vmserver
  distro                Distro. Valid options: ubuntu

*** Use vmbuilder <hypervisor> <distro> --help to get more options. Hypervisor, distro, and plugins specific help is only available when the first two arguments are supplied.

Options:
  -h, --help            show this help message and exit
  -d DESTDIR, --dest=DESTDIR
                        Specify the destination directory. [default:
                        <hypervisor>-<distro>].
  -c CONFIG, --config=CONFIG
                        Specify a additional configuration file
  --debug               Show debug information
  -v, --verbose         Show progress information
  -q, --quiet           Silent operation
  -t TMP, --tmp=TMP     Use TMP as temporary working space for image
                        generation. Defaults to $TMPDIR if it is defined or
                        /tmp otherwise. [default: /tmp]
  --templates=DIR       Prepend DIR to template search path.
  -o, --overwrite       Force overwrite of destination directory if it already
                        exist. [default: False]
  --in-place            Install directly into the filesystem images. This is
                        needed if your \$TMPDIR is nodev and/or nosuid, but
                        will result in slightly larger file system images.
  --tmpfs=OPTS          Use a tmpfs as the working directory, specifying its
                        size or "-" to use tmpfs default (suid,dev,size=1G).
  -m MEM, --mem=MEM     Assign MEM megabytes of memory to the guest vm.
                        [default: 128]
  --rootsize=SIZE       Size (in MB) of the root filesystem [default: 4096]
  --optsize=SIZE        Size (in MB) of the /opt filesystem. If not set, no
                        /opt filesystem will be added.
  --swapsize=SIZE       Size (in MB) of the swap partition [default: 1024]
  --raw=PATH            Specify a file (or block device) to as first disk
                        image.
  --part=PATH           Allows to specify a partition table in PATH each line
                        of partfile should specify (root first):
                        mountpoint size  one per line, separated by space,
                        where size is in megabytes. You can have up to 4
                        virtual disks, a new disk starts on a line containing
                        only '---'. ie:      root 2000      /boot 512
                        swap 1000      ---      /var 8000      /var/log 2000

  Network related options:
    --domain=DOMAIN     Set DOMAIN as the domain name of the guest. Default:
                        The domain of the machine running this script: .
    --ip=ADDRESS        IP address in dotted form [default: dhcp]
    --mask=VALUE        IP mask in dotted form [default: based on ip setting].
                        Ignored if --ip is not specified.
    --net=ADDRESS       IP net address in dotted form [default: based on ip
                        setting]. Ignored if --ip is not specified.
    --bcast=VALUE       IP broadcast in dotted form [default: based on ip
                        setting]. Ignored if --ip is not specified.
    --gw=ADDRESS        Gateway (router) address in dotted form [default:
                        based on ip setting (first valid address in the
                        network)]. Ignored if --ip is not specified.
    --dns=ADDRESS       DNS address in dotted form [default: based on ip
                        setting (first valid address in the network)] Ignored
                        if --ip is not specified.

ubuntu-vm-builder is Copyright (C) 2007-2008 Canonical Ltd. and written by
Soren Hansen <soren@canonical.com>.

As you can see there are a lot of options. Some of the more useful ones are:

--tmpfs
Allows you to specify a ram disk to build the image. This is much quicker than using the filesystem. If you have 1Gb RAM free you can specify --tmpfs - to use a ramdisk.
--part=PATH
For specifying a file outlining the partition table.

Partitioning

[straight from ubuntu docs]

Partitioning of the virtual appliance will have to take into consideration what you are planning to do with is. Because most appliances want to have a separate storage for data, having a separate /var would make sense.

In our case we will define a text file name vmbuilder.partition which will contain the following:

root 8000
swap 4000
---
/var 20000

Note

Note that as we are using virtual disk images, the actual sizes that we put here are maximum sizes for these volumes.

Our command line now looks like:

sudo vmbuilder kvm ubuntu --suite intrepid --flavour virtual --arch i386 -o --libvirt qemu:///system --mirror http://localhost:9999/ubuntu --tmpfs - --part vmbuilder.partition

User and Password

[straight from ubuntu docs]

Again setting up a virtual appliance, you will need to provide a default user and password that is generic so that you can include it in your documentation. We will see later on in this tutorial how we will provide some security by defining a script that will be run the first time a user actually logs in the appliance, that will, among other things, ask him to change his password. In this example I will use 'user' as my user name, and 'default' as the password.

To do this we use the following optional parameters:

  • --user USERNAME: Sets the name of the user to be added. Default: ubuntu.
  • --name FULLNAME: Sets the full name of the user to be added. Default: Ubuntu.
  • --pass PASSWORD: Sets the password for the user. Default: ubuntu.

Our resulting command line becomes:

sudo vmbuilder kvm ubuntu --suite intrepid --flavour virtual --arch i386 -o --libvirt qemu:///system --mirror http://localhost:9999/ubuntu --tmpfs - --part vmbuilder.partition --user user --name name --pass default

Installing Required Packages

Our virtual machine will provide SquirrelMail so we therefore require:

  • Apache
  • PHP
  • OpenSSH Server
  • SquirelMail

This is done using vmbuilder by specifying the --addpkg command multiple times:

--addpkg PKG
Install PKG into the guest (can be specfied multiple times)

However, due to the way vmbuilder operates, packages that have to ask questions to the user during the post install phase are not supported and should instead be installed while interactivity can occur. In the case of SquirelMail, the configuration of the IMAP and SMTP servers to connect to has to be done interactively, once the user logs in

Other packages that ask simple debconf question, such as mysql-server asking to set a password, the package can be installed immediately, but we will have to reconfigure it the first time the user logs in. Luckily, we don't have to worry about this for our example.

If some packages that we need to install are not in main, we need to enable the additional repositories using --comp and --ppa:

--components COMP1,COMP2,...,COMPN
A comma separated list of distro components to include (e.g. main,universe). This defaults to "main"
--ppa=PPA
Add ppa belonging to PPA to the vm's sources.list.

Limesurvey not being part of the archive at the moment, we'll specify it's PPA (personal package archive) address so that it is added to the VM /etc/apt/source.list, so we add the following options to the command line:

--addpkg apache2 --addpkg apache2-mpm-prefork --addpkg apache2-utils --addpkg apache2.2-common --addpkg libapache2-mod-php5 --addpkg php5-cli --addpkg squirelmail

OpenSSH

Another convenient tool that we want to have on our appliance is OpenSSH, as it will provide our admins to access to access the appliance remotely. However, pushing in the wild an appliance with a pre-installed OpenSSH server is a big security risk as all these server will share the same secret key, making it very easy for hackers to target our appliance with all the tools they need to crack it open in a breeze. As for the user password, we will instead rely on a script that will install OpenSSH the first time a user logs in so that the key generated will be different for each appliance. For this we'll use a --firstboot script, as it does not need any user interaction.

First Boot

As we mentioned earlier, the first time the machine boots we'll need to install openssh-server so that the key generated for it is unique for each machine. To do this, we'll write a script called boot.sh as follows:

# This script will run the first time the virtual machine boots
# It is ran as root.

perl -pi -w -e 's/127.0.0.1:9999/gb.archive.ubuntu.com/g;' /etc/apt/sources.list
apt-get update
apt-get install -qqy --force-yes openssh-server
locale-gen en_GB.UTF-8
/usr/sbin/update-locale LANG=en_GB.UTF-8
squirrelmail-configure
cp /etc/squirrelmail/apache.conf /etc/apache2/sites-available/squirrelmail
a2ensite squirrelmail
/etc/init.d/apache2 force-reload

The perl -pi -w -e 's/127.0.0.1:9999/gb.archive.ubuntu.com/g;' line replaces the mirror used for the installation with the correct version the guest should use for updated. The subsequent lines then configure and set up SquirrelMail. The script gets run during the first boot but not subsequently.

And we add the --firstboot boot.sh option to our command line.

First Login

Mysql and Limesurvey needing some user interaction during their setup, we'll set them up the first time a user logs in using a script named login.sh. We'll also use this script to let the user specify:

  • His own password
  • Define the keyboard and other locale info he wants to use

So we'll define login.sh as follows:

# This script is ran the first time a user logs in

echo "Your appliance is about to be finished to be set up."
echo "In order to do it, we'll need to ask you a few questions,"
echo "starting by changing your user password."

passwd

# give the opportunity to change the keyboard
sudo dpkg-reconfigure console-setup

echo "Your appliance is now configured.  To use it point your"
echo "browser to http://serverip/squirrelmail"

And we add the --firstlogin login.sh option to our command line.

Configuring Automatic Updates

To have your system be configured to update itself on a regular basis, we will just install unattended-upgrades, so we add the following option to our command line:

--addpkg unattended-upgrades

ACPI Event Handling

For your virtual machine to be able to handle restart and shutdown events it is being sent, it is a good idea to install the acpid package as well. To do this we just add the following option:

--addpkg acpid

Setting a Domain and Hostname

The domain and hostname can be set with the --domain and --hostname options respectively.

We add these options to the command:

--domain 3aims.com --hostname squirrelmail

Final Command

Here is what the command with all the options discussed above:

sudo vmbuilder kvm ubuntu --suite intrepid --flavour virtual --arch i386 -o --libvirt qemu:///system --tmpfs - --part vmbuilder.partition --user user --name name --pass default --addpkg apache2 --addpkg apache2-mpm-prefork  --addpkg apache2-utils --addpkg apache2.2-common --addpkg dbconfig-common --addpkg libapache2-mod-php5 --addpkg php5-cli --addpkg unattended-upgrades --addpkg acpid --addpkg squirrelmail --mirror http://mirroraddress:9999/ubuntu --firstboot boot.sh --firstlogin login.sh --domain 3aims.com --hostname squirrelmail

Specifying a vmbuilder Config File

The vm-builder program takes a lot of options for configuring the build. These can become a bit overwhelming so you can also use a config file and specify the -c argument to specify the config file vmbuilder should use. It will also look in some standard locations if no config file is specified.

Here's a suitable config file called squirrelmail.cfg to replace all the config options above.

[DEFAULT]
arch = i386
part = vmbuilder.partition
user = user
name = name
pass = default
tmpfs = -
firstboot = boot.sh
firstlogin = login.sh

[ubuntu]
mirror = http://127.0.0.1:9999/ubuntu
suite = intrepid
flavour = virtual
addpkg = apache2, apache2-mpm-prefork, apache2-utils, apache2.2-common, dbconfig-common, libapache2-mod-php5, php5-cli, squirrelmail, unattended-upgrades, acpid

[kvm]
libvirt = qemu:///system
hostname = squirrelmail
domain = 3aims.com

With this config file in place the final command looks like this:

sudo vmbuilder kvm ubuntu -c squirrelmail.cfg

Before you run it though you need to know about network configuration.

Network Configuration

Networking configuration is defined in three places:

The arguments you specify in the .cfg file or command line options to vmbuilder only affect the guest's /etc/network/interfaces file. The XML options in the VMBuilder/plugins/libvirt/templates/libvirtxml.tmpl file relataive to the directory in which you run the vmbuilder command specify the guest networking options and the XML options in /etc/libvirt/qemu/networks/default.xml specify the host networking.

The default.xml host networking template looks like this:

$ cat  /etc/libvirt/qemu/networks/default.xml
<network>
  <name>default</name>
  <uuid>7d237783-dc8d-4695-a0da-5cb60ccd40cc</uuid>
  <bridge name="virbr%d" />
  <forward/>
  <ip address="192.168.122.1" netmask="255.255.255.0">
    <dhcp>
      <range start="192.168.122.2" end="192.168.122.254" />
    </dhcp>
  </ip>
</network>

If you use the virsh shell console to dump the networking configuration you'll see the configuration actually used looks like this:

$ virsh -c qemu:///system net-dumpxml default
Connecting to uri: qemu:///system
<network>
  <name>default</name>
  <uuid>7d237783-dc8d-4695-a0da-5cb60ccd40cc</uuid>
  <forward mode='nat'/>
  <bridge name='vnet0' stp='on' forwardDelay='0' />
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254' />
    </dhcp>
  </ip>
</network>

I'm not sure why there is a discrepency but the default networking described by virsh is the one which is actually used. The libvirt documentation describes the XML configurations available. From http://libvirt.org/formatnetwork.html#examplesNAT it is clear that the above configuration is for NAT routing and that the bridge is named vnet0. You can confirm this by looking at other properties:

$ brctl show
bridge name     bridge id               STP enabled     interfaces
pan0            8000.000000000000       no
vnet0           8000.3e9bbdb7a9f7       yes             vnet1

$ sudo ifconfig

eth1      Link encap:Ethernet  HWaddr 00:1f:5b:84:23:e2
          inet addr:192.168.1.2  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::21f:5bff:fe84:23e2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:15061 errors:0 dropped:0 overruns:0 frame:24896
          TX packets:11167 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:17007964 (17.0 MB)  TX bytes:1294797 (1.2 MB)
          Interrupt:16

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:20228 errors:0 dropped:0 overruns:0 frame:0
          TX packets:20228 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:171245958 (171.2 MB)  TX bytes:171245958 (171.2 MB)

vnet0     Link encap:Ethernet  HWaddr 3e:9b:bd:b7:a9:f7
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          inet6 addr: fe80::8024:1bff:fe84:c44/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:437 errors:0 dropped:0 overruns:0 frame:0
          TX packets:490 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:77166 (77.1 KB)  TX bytes:102116 (102.1 KB)

So the networking configuration is NAT with a bridge, expecting guests to obtain an IP address via DHCP in the starting with 192.168.122.xxx. With this in mind we need to configure the guest to obtain an IP address via DHCP. This is easy to do by editing the guest's /etc/network/interfaces file to include these lines:

auto eth0 inetd dhcp

When you run the vmbuilder command without any other network options, this line is added automatically.

Building the Image

With all the changes in place let's build the image. You should have the following files in the current working directory:

$ ls -lah
total 32K
drwxr-xr-x  4 james james 4.0K 2009-03-09 20:00 .
drwxr-xr-x 39 james james 4.0K 2009-03-09 22:37 ..
-rw-r--r--  1 james james  225 2009-03-08 22:50 boot.sh
-rw-r--r--  1 james james  560 2009-03-09 00:31 lime.cfg
-rw-r--r--  1 james james  617 2009-03-09 15:09 login.sh
drwxr-xr-x  2 james james 4.0K 2009-03-09 00:37 ubuntu-kvm
drwxr-xr-x  3 james james 4.0K 2009-03-08 22:45 VMBuilder
-rw-r--r--  1 james james   36 2009-03-08 20:19 vmbuilder.partition

Now build the image:

sudo vmbuilder kvm ubuntu -c squirrelmail.cfg

Now start the new vm:

virsh -c qemu:///system start squirrelmail

Connect to the running VM with virt-manager and you'll see the boot.sh script has executed and that you are in the squirrelmail configuration application.

../2009/03/configure-squirrelmail.thumbnail.png

Choose option 2 and enter the correct details for your IMAP and SMTP servers then choose Q to quit, saving the changes.

Now you can login. Enter the username user and password default. Not the login.sh script runs, prompting you to change your password. Once changed, you are asked to enter it to get sudo access. You are then given an oppurtunity to configure your keyboard. Once the initrd image has been regenerated you will be left at a command prompt.

Logging in via SSH

Now that the first boot has been successful openssh-server will have been installed so you can now sign in over SSH. This is more convenient because you can more easily copy and paste from a terminal. Find out the IP address by typing:

ifconfig

The IP address is the IP labelled inet addr: in the eth0 section. In my case it is 192.168.122.19. You can now connect to the server from the host on that IP:

$ ssh user@192.168.122.19
The authenticity of host '192.168.122.19 (192.168.122.19)' can't be established.
RSA key fingerprint is 25:1f:53:a7:47:13:cd:cc:18:7e:bd:71:f1:4c:0c:34.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.122.19' (RSA) to the list of known hosts.
user@192.168.122.19's password:
Linux squirrelmail 2.6.27-11-server #1 SMP Thu Jan 29 20:19:41 UTC 2009 i686

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To access official Ubuntu documentation, please visit:
http://help.ubuntu.com/
Last login: Mon Mar  9 22:56:31 2009
user@squirrelmail:~$

The password to use is the one you set after the first login obviously.

Note

You will not be able to connect to the guest from the network because this IP is only available to the host and other guests on the host. To be able to access a guest from an external network you need to use bridged networking.

Testing SquirrelMail

Now that everything is set up you probably want to use SquirrelMail. The first thing to do is to visit the squirrelmail config page at http://192.168.122.19/squirrelmail/src/configtest.php to test the SquirrelMail configuration. Unfortunately, this can only be done from localhost or you'll get a 403 Permission Denied error so you need to install a command line browser.

On the server:

sudo apt-get install w3m

Now visit the page (taking note of the fact we have localhost in the URL instead of the IP address):

w3m http://localhost/squirrelmail/src/configtest.php

You should see all everything is OK, in which case you can start using SquirrelMail. Visit http://192.168.122.19/squirrelmail and login with your normal IMAP account login details.

../2009/03/squirrelmail.thumbnail.png

Congratulations, you've built, configured and are running your first useful JeOS.

If the computer hosting the virtual machine has more than one processor or has more spare memory it is easy to give the machine access to these extra resources. Just change the settings in the Virtual Machine Manager GUI tool.

More to come in Part 2...

(view source)

James Gardner: Home > Blog > 2009 > An Introduction to KVM and JeOS...