The FreeBSD Diary

This may well be the last article I write on this server. My existing development machine is still on 4.11. Today I will start the migration process to another server, which is already running 6.2-STABLE. In the process, I will add a RAID card to the server, and transfer the existing OS from a SATA drive to the RAID-1 cluster (based on two 160GB IDE drives).

Note that although I am using a RAID array here, it appears to the OS as a single HDD, so this process could equally apply to moving to a new bigger HDD. For your information, I will be using a 3Ware 7006-2 from 3Ware.

The main reason for putting RAID into this machine is that it will become my main development machine and take over the duties of two machines that now run FreeBSD 4.11 (which has recently hit End-of-Life and no more security updates will occur for 4.x).

We now join the migration, already in progress.

As of this point, I've installed the two new IDE drives and hooked them up to the 3Ware card. fter boot, I see this in the output from dmesg:

twe0: [GIANT-LOCKED]
twe0: 2 ports, Firmware FE7X 1.05.00.063, BIOS BE7X 1.08.00.048

That is my RAID card, just waiting to be set up. Good. Now I will reboot and go through the RAID BIOS and set up the cluster.

By the way, I've also installed sysutils/tw_cli and sysutils/3dm, two 3Ware utilities. I have mentioned them in a previous article on the 3Ware 9550SX card.

We will be using FreeSBIE to do the data transfer between the old and new HDD. Why? It is easier to copy an HDD when it is unused and with no chance of the data being modified. In short, the system is offline and copying the HDD is guaranteed to be correct.

From there, I used the instructions for Swapping boot drives around, written way back in August 1999. All I needed was the details for tar'ing up one file system and untar'ing it on the other. For this clean and easy technique, I will boot from something other than the SATA drive and the two IDE RAID drives. I could have used an HDD which already had FreeBSD on it. I choose instead to use a FreeSBIE live CD. FreeSBIE 2.0 gives me FreeBSD 6.2 and all the tools I need to massage the data according to my will. First, I boot up the system from the CD. Then I su to root, and move to my home directory. Then I create two directories:

mkdir new old

These two directories will form the starting point for my transfers. I will mount the existing SATA drive under the old directory and the new RAID-1 system under the new directory.

I need mount points for the various slices. So I did this:

cd old
mkdir root var tmp usr
cd ..
cd new
mkdir root var tmp usr

As you will see in the problems section later on, I forgot to do this, but you won't:

chmod 1777 ~/old/tmp

I started sysinstall and then used it to configure the new HDD (twed0). I click on the following links once in sysinstall:

• Configure | Fdisk | twed0 | A = Use Entire Disk | W = Write Changes | Yes | Standard Install a standard MBR (no boot manager)

I will now label the new HDD. Again, from the FreeBSD Configuration Menu, I click on Label. Ensure the right disk is selected, twed0 in my case.

Now I will create several slices and assign them some mount points. sysinstall will actually mount the slices at the mount points specified, and which we created earlier in this process. The short hand here is:

C = Create | 1 GB | FS | /mnt/root
C = Create | 2 GB | swap
C = Create | 2 GB | FS | /mnt/tmp
C = Create | 2 GB | FS | /mnt/var
C = Create | All (142GB) | FS | /mnt/usr

W = Write
Yes

Here is what the screen looked like after issuing the Write command.

FreeBSD Disklabel Editor

Disk: twed0     Partition name: twed0s1 Free: 0 blocks (0MB)

Part      Mount          Size Newfs   Part      Mount          Size Newfs
----      -----          ---- -----   ----      -----          ---- -----
twed0s1a  /mnt/root    1024MB UFS2+S Y
twed0s1b  swap         2048MB SWAP
twed0s1e  /mnt/tmp     2048MB UFS2+S Y
twed0s1f  /mnt/var     2048MB UFS2+S Y
twed0s1g  /tmp/usr      142GB UFS2+S Y

And in case it helps you, here are the the mounted filesystems:

[dan@ngaio:/mnt] $ mount
/dev/ad6s1a on / (ufs, local)
devfs on /dev (devfs, local)
/dev/ad6s1e on /tmp (ufs, local, soft-updates)
/dev/ad6s1f on /usr (ufs, local, soft-updates)
/dev/ad6s1d on /var (ufs, local, soft-updates)
fdescfs on /dev/fd (fdescfs)

polo:/usr/ports/distfiles on /usr/ports/distfiles (nfs)
/dev/twed0s1a on /mnt/root (ufs, local, soft-updates)
/dev/twed0s1e on /mnt/tmp (ufs, local, soft-updates)
/dev/twed0s1f on /mnt/var (ufs, local, soft-updates)
/dev/twed0s1g on /tmp/usr (ufs, local, soft-updates)
[dan@ngaio:/mnt] $

I will not be copying over devfs or fdescfs. NFS mounts should also not be copied over. In short, look only for the mount points associated with the HDD in question. In this case, ad6.

Of note were some interesting directories I had never seen before:

[dan@ngaio:/mnt] $ find .
.
./tmp
./tmp/.snap
./var
./var/.snap
./root
./root/.snap
./usr
[dan@ngaio:/mnt] $

What are those files? A Google found me a answer. In short, see man dump(8) and read the -L option.

Here is how I mounted the various slices of the SATA drive:

mount /dev/ad6s1a ~/old/root
mount /dev/ad6s1e ~/old/tmp
mount /dev/ad6s1f ~/old/var
mount /dev/ad6s1d ~/old/usr

Similarly, for the new RAID-1 cluster:

mount /dev/twed0s1d ~/new/root
mount /dev/twed0s1e ~/new/tmp
mount /dev/twed0s1f ~/new/var
mount /dev/twed0s1g ~/new/usr

Everything is ready. Let's transfer the data. I will do this one file system at a time. Let's start with the root partition:

tar --one-file-system -c -f - -C ~/old/root/ . | tar xpvf - -C ~/new/root/

The tar on the left will do a cd to ~/old/root/ and start tar'ing everything up, staying within the one file system. The -c is to create. The -f designates the output file, this case STDOUT.

The tar on the right will take extract, retain permissions, and display each file name as it is extracted. The -f - indicates STDIN as the source of the tarball. tar will do a cd to ~/new/root before it starts this process.

This process needs to be repeated for each mount point:

tar --one-file-system -c -f - -C ~/old/var/ . | tar xpvf - -C ~/new/var/
tar --one-file-system -c -f - -C ~/old/tmp/ . | tar xpvf - -C ~/new/tmp/
tar --one-file-system -c -f - -C ~/old/usr/ . | tar xpvf - -C ~/new/usr/

That sounds just too easy. Well, there is one gotcha. If anything refers to the old HDD by device, then it will break. You must change the new /etc/fstab before you reboot from the new RAID array. In my case, the file I need to edit is ~/new/root/etc/fstab and the changes required reflect what happened when I was using the Disk Label Editor. That is, I made sure that /dev/twed0s1a was mounted at /, twed0s1b was swap, twed0s1e was mounted at /tmp, etc.

Then I shutdown the system with a shutdown -p now and disconnected the SATA drive. During the power up process, I removed the FreeSBIE CD from the CD drive and let the system boot from the RAID card. Apart from the minor permission problem mentioned below, it went flawlessly.

The system would not boot from the RAID cluster the first time I tried. The problem was I had root on /dev/twed0s1d instead of /dev/twed0s1a. I went into FreeSBIE and rebooted. I went into the Disk Label Editor, deleted the slice, and created a new one, this time with s1a, not s1d. I had to redo the tar | tar process, but that didn't take too long. I also had to edit ~/new/root/etc/fstab to point at twed0 instead of ad6.

The next reboot worked, but there was a problem with /tmp permissions. A chmod 1777 fixed that. The next reboot showed no problems.