Checking for swappers on XenServer

This article will outline the steps I take to check for swappers on my XenServer and XenClassic setups.

XenServer

• Log into the huddle the host in question is located in
• Log into the host the slice/instances is located on
• Run iostat to check for any swappers on this host:

iostat -xkd 1

• If swappers are found, cat the device of the slice to get its minor number:

cat /sys/block/tdk/dev  # returns something like: 1234:20

• Get the UUID for this slice

tap-ctl list | grep minor=20

• Check if this slice is a "swap" partition (make sure it is _not_ a root partition!):

xe vdi-list uuid=xxxx-xxxx-xxxx-xxxx

• Check to make sure no other tasks are currently being perform on this host

xe task-list

• Now force a reboot

xe vm-reboot --force name-label=slice01234

• Check that the slice has actually rebooted:

xe vm-list name-label=slice01234 params=start-time

• Verify that the output from the last command is within the current timestamp

date

That's it! You have successfully killed a swapper and brought the slice back up to a normal load.

XenClassic

This is how you do the above on a XenClassic setup.

• Log into the huddle the host in question is located in
• Log into the host the slice/instances is located on
• Run iostat to check for any swappers on this host:

iostat -xkd 1

• If swappers are found, list the device mapper for this slice:

ls -l /dev/mapper/ | grep ' 20 '

• Now "destroy" that slice (we are not permanently destroying this device, that's just the terminology):

xm destroy slice01234

• Finally, re-create this slice (this takes information from a configuration flatfile found under /etc/xen/slice01234):

xm create slice01234

That's it! You have successfully killed a swapper and brought the slice back up to a normal load.

We can accomplish the above with a single script (well, a series of CLI calls) like so:

(echo "device name tps MB_read/s MB_write/s MB_total/s";\
(ls -l /dev/mapper/; iostat -m 1 2) |\
awk 'BEGIN {section=0} {\
  if($3=="root") devices[$6]=$9;\
  if($1=="Device:") section++;\
  else if(section==2 && $0) {\
    dev=$1; tps=$2; read=$3; write=$4;\
    if(substr(dev,1,2)=="dm") {split(dev,parts,"-"); name=devices[parts[2]]} \
    if(!name) name="unknown";\
    print dev" "name" "tps" "read" "write" "read+write}}' |\
sort -k3n) | column -t

And, if the above command(s) return a slice id, then execute:

xm destroy slice01234 && xm create slice01234 && xm console slice01234

XenClassic slice configuration file

Below is an example of what a slice configuration flatfile (which is, by default, located at /etc/xen/slice01234) would look like:

name="slice01234"
memory=512
vcpus=4
kernel="/etc/xen/seeds/01/vmlinuz-2.6.35.4-generic"
ramdisk="/etc/xen/seeds/01/initrd.img-2.6.35.4-generic"
vif=['bridge=eth0, ip=xxx.xxx.xxx.xxx, mac=4f:d0:c4:10:ab:0c','bridge=eth1, ip=10.x.x.x, mac=4f:e0:29:9f:c3:fe']
disk=[ 'phy:slices/slice01234_root,sda1,w', 'phy:slices/slice01234_swap,sda2,w' ]
root="/dev/sda1 ro"
cpu_weight=512
extra="xencons=tty console=tty1 clocksource=acpi_pm "