Xen is my favorite hypervisor for running guests on Debian system (I mean a a dom0). I use it fo quite a while now and would like to share some tweak I’ve come up with over times.
Xen is robust, high performance and stable hypervisor… If you know what you are doing and why you do it this way! Here follows some hints grabbed since my first steps with Xen in 2006 (thanks @pierrotcarre for bringing me to Xen).
- No services on your dom0
The hypervisor’s job is just crucial: It takes care of guest accesing needed ressources with respect to the configuration setup by the admin. By itself, this task isn’t very ressource consumming but it is mandatory for stable and efficient virtualisation infrastructure! Do not disturb your hypervisor with secondary tasks it should not care about. This is mostly a way to introduce bugs, security issue or performance loss. I more particularily think oabout setup where the dom0 is asked to manage storage and replicate data with some very complex solutions such as DRBD. Invest in SAN, it definatly worth it - Use dom0_mem
This parameter allow the hypervisor to handle the minimum memory needed for it own life. If you respect the former rule you shouldn’t need more than 512MB for a Debian based dom0 (Leeny or Squueeze). This parameter can be tweaked regarding your distribution requirement and the packages that comes with it “out of the box”. - Dom0 pooling
One of the most attractive feature of virtualisation remains migration, or vmotion, or whatever makes a VM able to move from one host to another. In order to achieve this, you’ll need to conform to some rules.
The firs t and most obvious one is about virtual disks. Both source and destination dom0 need to have access to the virtual disks of the VM at the time of migration. But you can extend this to all the ressources needed by the VM to run. As such, network interfaceshave to be present on both dom0s, so the VM can be plugged to the network on the right bridge (for example). Some admin might forget but this apply to CPU too in an even more subtile maner. VMs are granted vcus that embend the features available on the dom0 CPUs. Then you should always take care of moving a virtual machine to a host that have the same capabilities than the source host (sould mean the same CPU…. or very similar). Imagine you are skateborading and you suddently forget how to skate… My two cents you’ll fall or worst! Generally speaking, Xen hosts should be organise in pools of servers, based on the disks and network ressources they can access and on their “CPU profil”. - Jet lag makes your VMs sick!
When migrating 2 VMs from one host to another, ensure the 2 hosts are in sync!. This simple advice will avoid many weired problems and is simple to apply by using ntpd on every dom0. Don’t expect your VMs to properly run if time changes every time it migrates!
Sometimes ntpd is notenough. From my experience I saw Debian Lenny domU/dom0 had problem with keeping in sync. The following parameters, apllied to the domU config file may helpextra='clocksource=jiffies'
You must also set it on the dom0 using:
echo "jiffies"> /sys/devices/system/clocksource/clocksource0/current_clocksource
Jiffies clocksources are considred a poor choice and should only be used as fallback. As a conséquenc never use this unless you see this kind of messages:
clocksource/0: Time went backwards
The Xen Debian Wiki also report another workaround that do not use jiffies, and so mshuold be preferred. I have nerver used it but you can find more about it here http://wiki.debian.org/Xen#Possible_problems_and_bugs-1 workaround#3 - PyGrub helps!
PyGrub is not really a bootloader. It’s just a tool that will lokk into your VM filesystem to get grub config files and findout which kernel, initrd, serial console, etc… to use. It will then copies necessary files to your dom0 and create dynamicly an vm specification file. This allow you to boot your PV VM with their own kernelif they embend domU xen capabilities boot (most of nowadays kernel support this). You can then use your favorite package manager to keep your kernel and modules updated on the VM and don’t worry anymore of what your dom0 know about domU kernels.
Please note that PyGrub need to access the filesystem of your VM and so have support for it. A time of writing, PyuGrub supports ext2,ext3 andext4. A patch is also available for XFS: http://xen.1045712.n5.nabble.com/XFS-support-for-pygrub-patch-td3281142.html#a4387931 - bridged networks
Here is a recurrent topic in Xen mailing lists! Whereas the configuration is very simple, you should not use Xen defaults. If you wish to use the bridged network mode of Xeb (VMs access the physical network without routing or NAT), Xen comes with thenetwork-bridgescript (in /etc/xen/xen-config.sxp). Let me be clear: this awful script is just awful is absolute non-sense! most things it does can be done in cleaner way, and the other things just don’t need to be done. This script will also be remove from Xen in the future release (if not already removed in 4.1.)
So if you want bridged networks for your VMs on your debian host, just trust ifupdown! It’s the historical network manager in debian!It very feature richand allow for advanced configurations just modifying /etc/network/interfaces.
Basicly to grant access to your VMs to the physical network on eth0 just use:
auto xenbr0Network is on tagged VLAN?
iface xenbr0 inet static
bridge_ports eth0
bridge_maxwait 5
bridge_fd 0
auto xenbr0
iface xenbr0 inet static
bridge_ports eth0.100
bridge_maxwait 5
bridge_fd 0You care about redondancy?
auto bond0auto xenbr0
iface bond0 inet manual
slaves eth0 eth1
iface xenbr0 inet static
bridge_ports bond0.100
bridge_maxwait 5
bridge_fd 0Don’t forget to tell Xen to mind his own business. In /etc/xen/xen-config.sxp
(network-script network-dummy)And ensure the script exists and does what it has to do: nothing!
cat /etc/xen/scripts/network-dummy
#!/bin/sh
exit 0 - VNC consoles
The Xen console t oaccess VM shell is very handy (and has really improved since the 2.X versions of Xen). With that console you can for example, copy/paste strings, wich is mandatory for lazy admins like me. Unfortunately the most commonly used console it graphical tools (Xen Center, Convirt etc…) is the VNC one. Then a good setup would be to have both Xen console (xm) and VNC.
In order to achieve this you’ll have to configre VMs to use virtual framebuffer device (for both PV ou HVM) and modify their inittab file.
Adding a framebuffer device can be done in the VM config file:
vfb=['type=vnc]
Ouch you may need a rest now! Or you may take a look at the vnc related options (man xmdomain.cfg)
The inittab modifications are quite simple too. just check you have the following lines in /etc/inittab
1:2345:respawn:/sbin/getty 38400 hvc0
2:23:respawn:/sbin/getty 38400 tty1
This way a serial console will be available usingxm consoleor using VNC.
Note that daemon startup messages will be printed to the firs defined console in inittab.
Leave a Reply