How to extend a file system using LVM/XEN

You have a nifty XEN domU running on top of an LVM backed dom0. You run out of space, and you need to extend. It’s not hard, but it can be tedious if you don’t want to the domU down.

The hardest part is making sure that there are no processes using the partition you want to extend, as you have to umount it. While ext2online (available in RH kernels, but not, as of 2.6.18, stock) permits you to expand the file system, unfortunately, with xen 3.1.0, extending the Logical Volume underneath does not communicate the new length to the domU.

Usually, people miss the NFS exports. To unexport temporarily (which does screw any client using it), you can use exportfs. You need to give it the IP addresses (possibly with netmasks) that you wish to remove. They have to match. This is still easier than running exportfs -a -u, editing the /etc/exports and running exportfs -a, as it doesn’t screw quite so many NFS clients.

In these examples, tbm800 is a guest domU, which happens to also be an NFS server. airbus is the dom0.

tbm800-[~] root 1 #df -H /proj/thintropy/skeletons Filesystem Size Used Avail Use% Mounted on /dev/sde5 11G 8.0G 2.1G 80% /proj/thintropy/skeletons tbm800-[~] root 7 #umount /proj/thintropy/skeletons umount: /proj/thintropy/skeletons: device is busy umount: /proj/thintropy/skeletons: device is busy tbm800-[~] root 9 #exportfs -u tbm800-[~] root 10 #umount /proj/thintropy/skeletons

Now, we need to know what the backend block device which was used to mount the device. Unfortunately, xm block-list does not show us anything useful, but it’s in the logs:

airbus-[/var/log/xen] root 10 #grep tbm800 xend.log | grep skeletons | grep DevController [2007-10-09 09:32:02 xend 3682] DEBUG (__init__:1072) DevController: writing {'domain': 'tbm800', 'frontend': '/local/domain/1/device/vbd/2117', 'dev': 'sde5', 'state': '1', 'params': '/dev/AirbusGroup0/tbm800skeletons', 'mode': 'w', 'online': '1', 'frontend-id': '1', 'type': 'phy'} to /local/domain/0/backend/vbd/1/2117. [2007-10-09 10:43:49 xend 27521] DEBUG (__init__:1072) DevController: writing {'domain': 'tbm800', 'frontend': '/local/domain/14/device/vbd/2117', 'dev': 'sde5', 'state': '1', 'params': '/dev/AirbusGroup0/tbm800skeletons', 'mode': 'w', 'online': '1', 'frontend-id': '14', 'type': 'phy'} to /local/domain/0/backend/vbd/14/2117.

Note the 2117. We need to know the domain-ID, as well.

airbus-[/var/log/xen] root 11 #xm list tbm800 Name ID Mem(MiB) VCPUs State Time(s) tbm800 14 192 1 -b---- 7812.5 airbus-[/var/log/xen] root 12 #xm block-detach Error: 'xm block-detach' requires 2 arguments. Usage: xm block-detach Destroy a domain's virtual block device. [1] 31565 exit 1 xm block-detach airbus-[/var/log/xen] root 13 #xm block-detach 14 2117 </example> Now, we can manipulate the LV: airbus-[/var/log/xen] root 14 #lvextend -L +10G /dev/AirbusGroup0/tbm800skeletons Extending logical volume tbm800skeletons to 20.00 GB Logical volume tbm800skeletons successfully resized Next we can attach the block device again. We need to know some details, but they are in the config file: airbus-[/var/log/xen] root 15 #grep tbm800skeletons /etc/xen/tbm800 'phy:/dev/AirbusGroup0/tbm800skeletons,sde5,w' airbus-[/var/log/xen] root 16 #xm block-attach Error: 'xm block-attach' requires between 4 and 5 arguments. Usage: xm block-attach Create a new virtual block device. [1] 31646 exit 1 xm block-attach airbus-[/var/log/xen] root 17 #xm block-attach 14 phy:/dev/AirbusGroup0/tbm800skeletons sde5 w </example> On the domU, again, now we can resize things: tbm800-[~] root 12 #e2fsck -f /dev/sde5 e2fsck 1.40-WIP (14-Nov-2006) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information /dev/sde5: ***** FILE SYSTEM WAS MODIFIED ***** /dev/sde5: 323289/1310720 files (1.4% non-contiguous), 1979906/2621440 blocks And finally, the resize operation: tbm800-[~] root 14 #resize2fs /dev/sde5 resize2fs 1.40-WIP (14-Nov-2006) Resizing the filesystem on /dev/sde5 to 5242880 (4k) blocks. The filesystem on /dev/sde5 is now 5242880 blocks long. tbm800-[~] root 15 #mount -a tbm800-[~] root 16 #df -H /proj/thintropy/skeletons Filesystem Size Used Avail Use% Mounted on /dev/sde5 22G 8.0G 13G 40% /proj/thintropy/skeletons tbm800-[~] root 17 #exportfs -a The device is now 22G in size.