Laptop repartitioning

When my Toshiba laptop died, I knew that the hard disk I had was still good, and so I got the UltraBay HD adapter to go with my IBM Thinkpad. When the Thinkpad arrived, I declined the MS EULA that the laptop booted up with, turned it off again, and swapped in my old HD. Thanks to the fact that Linux (and NetBSD) do not have stupidly motherboard specific kernels (it’s EVEN WORSE with Vista. See: http://ars.userfriendly.org/cartoons/?id=20061106&mode=classic ) I can just move my hard disk to a new system.

Anyway, in fall 2005, when my motherboard blew up, it took my hard disk as well (I’d reformatted the IBM HD, and was now using that). I replaced the hard disk, and still have to RMA the original, and wound up with a 60G drive inside the laptop, and a 40G drive in the bay. Most of the time, this is the right combination, with the CDrom in the docking station. The I/O path to the ultrabay is a bit faster than the I/O path to the docking station, although you can’t boot from a cdrom in the docking station.

The question arises: where do you put your music? When I’m at my home office, the music is accessed over HTTP, so I don’t need a local copy. When I’m travelling, I’m often on battery. I’d prefer to have my spare UltraBay battery in the bay rather than my spare disk. On sensible airplanes, and on VIA rail, you get power, so you don’t need the battery. I think that I’ve now concluded that putting my music on my second disk is okay, but up to now, I haven’t done that.

The second problem is where to put the master Linux Kernel git tree. Until this past summer, with “git clone -s” (more on that later), one wanted to have kernel git trees on the same partition as one’s experimental trees, since they could share with hard links. I made the decision that if I was doing core kernel development (vs Openswan + UML), I probably had test machines, so I have power. So I put my kernel in /mara1/git.

The result is that up to today I had five ~8G partitions on my external drive. This is big enough to be useful for work like git, but small enough that I won’t loose too much at a time, and I can do backups reasonably easily. Generally, I like to either backup everything on a partition, or nothing. So if I have data on a partition which can trivially be recovered from the internet, I may not back it up.

This means that my laptop (whose name is “marajade”) looks like:

PRIMARY DISK / + /usr - operating system /boot - kernels (100M) /home - my home directory /var - logs and administrative files /corp - my corporate data, projects, etc. /sandel - local programs, source code and customization /mara6 - project space (mostly openswan + UML as it turns out) /mara7 - customer project space (some kernels) /mara8 - ccache, spam folders, music SECONDARY DISK /mara1 - many git trees /mara2 - more openswan + other stuff sandboxes /mara3 - random data /mara4 - freebsd and other code, more customer project space /mara5 - backups... never really used.

The problem with any partitioning scheme other than “put everything in one partition”, is that one can run out of space on one partition, and still have space elsewhere. This is very difficult for new users to deal with, but it can be very annoying for experienced users as well.

Of course, with everything in one partition there are other problems: you can have logs run away and eat ALL the disk space, and then you may have a problem, and you may not notice for perhaps even YEARS. If you have a file system error, you are out of luck. If you re-install your operating system, you likely LOOSE a lot of your data.

I had my secondary disk partitions mounted via the LABEL= hack so that they would be found whether the second disk was hdc (ultrabay), or hde (docking station). I marked them noauto, so if they aren’t present life doesn’t end, and have a boot script that runs mount on them instead.

After using the Logical Volume Manager tools a bunch to deal with Xen, I realized that I could have the best of both solutions. I wondered at first, if I could create a single volume group, and sometimes not have one physical volume, and perhaps could simply force LVs to be on one physical volume or another, and migrate stuff around as I need it. That doesn’t work, unfortunately, see: http://www.sandelman.ca/mcr/blog/2006/05/24#how_to_unscrew_your_lvm

My solution is two volume groups, and I was able to move data around a bit to free up a partition to make my first physical volume for each volume group, and then restore data. The end result is that I now have a /corp/projects/SOMETHING (primary disk), and /corp/projects/SOMETHING2 (secondary disk), for a lot of project related data. Note that I didn’t move my operating system or personal data. I’m still a bit hesistant about LVM, and in particular, my local kernel source (which I’d need to build LVM modules!) is accessible without LVM.

My new /etc/fstab:

/dev/hda5 / ext3 defaults,errors=remount-ro 0 1 /dev/hda1 /boot ext2 defaults 0 2 /dev/hda12 /corp ext3 defaults 0 2 /dev/hda8 /home ext3 defaults 0 2 /dev/hda10 /mara6 ext3 defaults 0 2 /dev/hda9 /sandel ext3 defaults 0 2 /dev/hda6 /usr ext3 defaults 0 2 /dev/hda7 /var ext3 defaults 0 2 LABEL=UmlRoot /uml ext3 defaults 0 0 LABEL=CACHE /cache ext3 defaults 0 0 LABEL=DistrosKernel /distros/kernel ext3 defaults 0 0 LABEL=ProjectOpenswan /corp/projects/openswan ext3 defaults 0 0 LABEL=ProjectXelerance /corp/projects/xelerance3 ext3 defaults 0 0 LABEL=ProjectCobbix /corp/projects/cobbix ext3 rw,noauto,user,exec,defaults 0 0 LABEL=ProjectVpnx /corp/projects/vpnx ext3 rw,noauto,user,exec,defaults 0 0 LABEL=ProjectFreeradiu /corp/projects/freeradius ext3 rw,noauto,user,exec,defaults 0 0 LABEL=ProjectMisc /corp/projects/Misc ext3 rw,noauto,user,exec,defaults 0 0 LABEL=UmlRoot2 /uml2 ext3 rw,noauto,user,exec,defaults 0 0 LABEL=ProjOpenswan2 /corp/projects/openswan2 ext3 rw,noauto,user,exec,defaults 0 0 LABEL=ProjectsHifn2 /corp/projects/hifn2 ext3 rw,noauto,user,exec,defaults 0 0 LABEL=ProjectCroquet /corp/projects/opencroquet ext3 rw,noauto,user,exec,defaults 0 0 lvm> vgs VG #PV #LV #SN Attr VSize VFree MaraMainDisk 3 5 0 wz--n- 18.50G 4.50G MaraUltraBay 5 8 0 wz--n- 35.39G 17.42G lvm> lvs LV VG Attr LSize Origin Snap% Move Log Copy% CACHE MaraMainDisk -wi-ao 3.00G DistrosKernel MaraMainDisk -wi-ao 4.00G ProjectOpenswan MaraMainDisk -wi-ao 3.00G ProjectXelerance MaraMainDisk -wi-ao 2.00G UMLROOT MaraMainDisk -wi-ao 2.00G ProjectCobbix MaraUltraBay -wi-ao 1000.00M ProjectCroquet MaraUltraBay -wi-ao 2.00G ProjectVpnx MaraUltraBay -wi-ao 2.00G ProjectsFreeradius MaraUltraBay -wi-ao 1.00G ProjectsHifn2 MaraUltraBay -wi-ao 6.00G ProjectsMisc MaraUltraBay -wi-ao 2.00G ProjectsOpenswan2 MaraUltraBay -wi-ao 2.00G UMLROOT2 MaraUltraBay -wi-ao 2.00G marajade:~# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/hda5 1829159 458651 1272915 27% / udev 10240 96 10144 1% /dev devshm 255272 0 255272 0% /dev/shm /dev/hda1 112883 61155 45706 58% /boot none 65536 160 65376 1% /tmp none 65536 0 65536 0% /ramdisk none 4807056 704176 3858696 16% /var/run /dev/hda12 7692876 6294116 1007984 87% /corp /dev/hda8 7692876 2754784 4547316 38% /home /dev/hda9 7692876 2200808 5101292 31% /sandel /dev/hda6 7692876 2652548 4649552 37% /usr /dev/hda7 4807056 704176 3858696 16% /var /dev/mapper/MaraUltraBay-ProjectCobbix 991949 627375 313374 67% /corp/projects/cobbix /dev/mapper/MaraUltraBay-ProjectsMisc 2064208 828640 1130712 43% /corp/projects/Misc /dev/mapper/MaraUltraBay-ProjectsFreeradius 1032088 102260 877400 11% /corp/projects/freeradius /dev/mapper/MaraUltraBay-ProjectsOpenswan2 2064208 301440 1657912 16% /corp/projects/openswan2 /dev/mapper/MaraUltraBay-UMLROOT2 2064208 217764 1741588 12% /uml2 /dev/mapper/MaraMainDisk-CACHE 3096336 1927832 1011220 66% /cache /dev/mapper/MaraUltraBay-ProjectVpnx 2064208 979912 1000412 50% /corp/projects/vpnx /dev/mapper/MaraUltraBay-ProjectCroquet 2064208 136516 1822836 7% /corp/projects/opencroquet /dev/mapper/MaraMainDisk-UMLROOT 2064208 217768 1741584 12% /uml /dev/mapper/MaraMainDisk-ProjectOpenswan 3096336 2048528 890524 70% /corp/projects/openswan /dev/mapper/MaraMainDisk-ProjectXelerance 2047379 1902 1961591 1% /corp/projects/xelerance3 /dev/mapper/MaraUltraBay-ProjectsHifn2 6192704 4020112 1983852 67% /corp/projects/hifn2 /dev/mapper/MaraMainDisk-DistrosKernel 4128448 2743844 1342664 68% /sandel/distros/kernel

What would be nice

It would be neat if there was a way to move a logical volume from one volume group to another, and do this in the background. I would do this by turning the logical volume into a RAID-1 ( http://en.wikipedia.org/wiki/RAID ), and letting the RAID-1 mirror the data. The trick would be to do this using the dm-mapper, changing the “plumbing” underneath, and when it was done, remove the RAID-1 settings, leaving the file system layer ignorant of the change.

It would also be nice to be able to resize the partitions without unmounting them. I think that one of XFS or ReiserFS-4 can do this. I use ext3 at present.

How I use git-clone

The new thing about git clone -s, is that it doesn’t use hard links, or soft-links, but instead sets up .git/objects/info/alternatives. This effectively reduces the cost of a kernel .git tree to the space for the checked out files (about 50M for Linux) and objects. I move my “Linus Torvalds” git tree to =/distros/kernels/linus.git=, and git clone it. I put that master tree on my primary disk, because then I always have it, while I can put as many kernels with object files on per-project places as I go.

(=/distros/kernels= is an example of stuff that I wouldn’t bother backing up, btw.)

It is my intention to even create partitions for specific projects as I need them, and remove them when I am done. The question will be, how in such a dynamic world, will I get all my backups to do the right thing.