Things look quiet here. But I've been doing a lot of blogging at
dan.langille.org because I prefer WordPress now.
Not all my posts there are FreeBSD related.
I am in the midst of migrating The FreeBSD Diary over to WordPress
(and you can read about that here).
Once the migration is completed, I'll move the FreeBSD posts into the
new FreeBSD Diary website.
vinum - A simple install to improve performance10 August 1999
The Vinum Volume Manager is a block device driver which implements virtual disk drives.
It isolates disk hardware from the block device interface and maps data in ways which
result in an increase in flexibility, performance and reliability compared to the
traditional slice view of disk storage. Vinum implements the RAID-0, RAID-1 and RAID-5
models, both individually and in combination.
The vinum home page is at http://www.vinumvm.org/.
There is also lost article by Jake Burkholder
which I used for the basis of this exercise. You should read
vinum(4) and
vinum(8) and before continuing with this article. It explains the concepts and
terms used within this article (such as plexes, subdisks, and volumes) and which are
necessary if you wish to implement vinum.
Well, in my case, it was to create a larger volume and to increase performance.
The box I was working on was destined to become the www and cvsup mirror for New Zealand FreeBSD User Group. But the box
had only 1GB. So I removed those two 500MB drives and added in three 1GB drives.
You may wish to read how I did the drive swap.
If you wish to learn more about striping and RAID, please read http://www.lemis.com/vinum.html as I will not
cover those topics at all.
The first step was to figure out what size my disks were. My first attempts
involved assuming they were both 1GB drives. There weren't. And this caused
problems. So here's how to find out the size. The drives I'm using are SCSI
drives da1 and da2.
# fdisk da1
******* Working on device /dev/rda1 *******
parameters extracted from in-core disklabel are:
cylinders=496 heads=118 sectors/track=35 (4130 blks/cyl)
parameters to be used for BIOS calculations are:
cylinders=496 heads=118 sectors/track=35 (4130 blks/cyl)
Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165,(FreeBSD/NetBSD/386BSD)
start 0, size 2051000 (1001 Meg), flag 80 (active)
beg: cyl 0/ sector 1/ head 0;
end: cyl 342/ sector 35/ head 117
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
<UNUSED>
# fdisk da2
******* Working on device /dev/rda2 *******
parameters extracted from in-core disklabel are:
cylinders=4085 heads=14 sectors/track=36 (504 blks/cyl)
Figures below won't work with BIOS for partitions not in cyl 1
parameters to be used for BIOS calculations are:
cylinders=4085 heads=14 sectors/track=36 (504 blks/cyl)
Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165,(FreeBSD/NetBSD/386BSD)
start 0, size 2059140 (1005 Meg), flag 80 (active)
beg: cyl 0/ sector 1/ head 0;
end: cyl 953/ sector 36/ head 13
The data for partition 2 is:
<UNUSED><unused>
The data for partition 3 is:
<UNUSED><unused>
The data for partition 4 is:<unused>
<UNUSED>
If you look above, you'll find that da1 is 1001MB and da2 is
1005MB.
Disk partitions
I am assuming your have your disk partitions and slices ready to go. If you
don't, please see Formatting Media For Use With FreeBSD
but for your reference, my two disk are "dangerously" dedicated.
Addenda - 26 August 1999
When I originally wrote the section on File System Types, my default editor was vi.
Then a reader sent me a message telling me telling me that that various UNIX
commands use the EDITOR command. Thus, you can set your default editor for various
commands. I thought that was brilliant! Of course I rushed off to write a new
article about that, only to find I'd already written one and forgotten about it!
vinum only works on vinum type file systems. By default, the
file systems are of type unused. You can change the file system type (fstype) to
"vinum" by using disklabel.
# disklabel -e da1
If you know how to use vi, then go ahead and ignore the following
instructions. Otherwise this should help a big help:
use the arrow keys to position yourself at the start of where it says unused
under the fstype column.
press x to delete the existing fstype entry
press i to start insert
type vinum
press escape to end insert mode
press :wq, then enter to save your changes and exit
The process of creating a vinum volume will erase the data which exists on
the file system. If you are converting existing disks to use vinum, back up
your data first. You can restore the data later.
Create the configuration file
WARNING: Grey Lehey has said not to create striped plexes with a power of 2 stripe size.
DO NOT USE THE 256k VALUES I USED.
As mentioned in Jake's article, I used /etc/vinum.conf as my configuration
file. Here is what it contains:
drive d1 device /dev/da1e
drive d2 device /dev/da2e
volume test
plex org striped 256k <==== do not use a power of 2 like I did
sd length 1001m drive d1
sd length 1001m drive d2
See the vinum documentation for more information. The new volume device I am
creating will be called test. Also note that I am striping and specifying
the same size (1001MB) for both drives. This is despite that fact that one drive
actually contains 1005 MB. See below for more information on the error messages I
encountered when I didn't specify the correct sizes.
man vinum also contains a "HOW TO SET UP VINUM" section which
gives various options. Please refer to that when deciding what to put into your
configuration file.
Create the vinum devices
vinum uses it's own set of devices to implement what it needs to do what it
does best. These devices are located in /dev/vinum. To create
these devices, here's what I did:
# vinum create -f /etc/vinum.conf
Configuration summary
Drives: 2 (4 configured)
Volumes: 1 (4 configured)
Plexes: 1 (8 configured)
Subdisks: 2 (16 configured)
D d1 State: up Device /dev/da1e Avail: 0/1001 MB (0%)
D d2 State: up Device /dev/da2e Avail: 4/1005 MB (0%)
V test State: up Plexes: 1 Size: 2002 MB
P test.p0 S State: up Subdisks: 2 Size: 2002 MB
S test.p0.s0 State: up PO: 0 B Size: 1001 MB
S test.p0.s1 State: up PO: 256 kB Size: 1001 MB
Creating the new file system
The next step is to create the new file system. That's really easy actually.
# newfs -v /dev/vinum/test
newfs: /dev/vinum/test: not a character-special device
/dev/vinum/test: 4100096 sectors in 1001 cylinders of 1 tracks,
4096 sectors
2002.0MB in 63 cyl groups (16 c/g, 32.00MB/g, 7936 i/g)
super-block backups (for fsck -b #) at:
32, 65568, 131104, 196640, 262176, 327712, 393248, 458784,
524320, 589856, 655392, 720928, 786464, 852000, 917536, 983072,
etc...
The -v option is very interesting:
-v Specify that the disk does not contain any partitions, and that
newfs should build a file system on the whole disk. This option
is useful for synthetic disks such as vinum.
Creation errors
If you don't change your fstype to vinum, you will get the following error when you
try to create.
1: drive d1 device /dev/da1e
** 1 Drive d1 has invalid partition type: Inappropriate file
type or format
2: drive d2 device /dev/da2e
** 2 Drive d2 has invalid partition type: Inappropriate file
type or format
Always check /var/log/messages after doing a vinum create.
When I used the following configuration, I encountered my first problem:
sd length 1g drive d1
sd length 1g drive d2
But this configuration resulted in the following:
# vinum create -f /etc/vinum.conf
7: sd length 1g drive d1
** 7 No space for test.p0.s0 on d1: No space left on device
8: sd length 1g drive d2
** 8 Unnamed sd is not associated with a plex: Invalid argument
Configuration summary
Drives: 2 (4 configured)
Volumes: 1 (4 configured)
Plexes: 1 (8 configured)
Subdisks: 0 (16 configured)
D d1 State: up Device /dev/da1e Avail: 1001/1001 MB (99%)
D d2 State: up Device /dev/da2e Avail: 1005/1005 MB (99%)
V test State: down Plexes: 1 Size: 1024 MB
P test.p0 S State: faulty Subdisks: 1 Size: 1024 MB
S test.p0.s0 State: down PO: 0 B Size: 1024 MB
The log messages for this are:
Aug 9 20:30:45 fred /kernel: vinum: drive d1 is up
Aug 9 20:30:45 fred /kernel: vinum: drive d2 is up
Aug 9 20:30:45 fred /kernel: vinum: plex test.p0 does not have at
least 2 subdisks
Aug 9 20:30:45 fred last message repeated 2 times
Aug 9 20:30:45 fred /kernel: vinum: test.p0 is faulty
So then I changed the configuration to:
sd length 1001m drive d1
sd length 1005m drive d2
And I tried again:
# vinum create -f /etc/vinum.conf
Configuration summary
Drives: 2 (4 configured)
Volumes: 2 (4 configured)
Plexes: 2 (8 configured)
Subdisks: 2 (16 configured)
D d1 State: up Device /dev/da1e Avail: 0/1001 MB (0%)
D d2 State: up Device /dev/da2e Avail: 0/1005 MB (0%)
V test State: down Plexes: 2 Size: 2006 MB
P test.p0 S State: faulty Subdisks: 1 Size: 1024 MB
P test.p1 S State: down Subdisks: 2 Size: 2006 MB
S test.p0.s0 State: down PO: 0 B Size: 1024 MB
S test.p1.s0 State: stale PO: 0 B Size: 1001 MB
S test.p1.s1 State: stale PO: 256 kB Size: 1005 MB
Well, that looks much better! We're making progress. At least, that's what
I thought. Note the faulty and down entries in the
above. And if you check the logs, you'll find.
Aug 9 20:34:28 fred /kernel: vinum: plex test.p0 does not have at
least 2 subdisks
Aug 9 20:34:28 fred /kernel: vinum: test.p1 must have
equal sized subdisks
Aug 9 20:34:28 fred /kernel: vinum: test.p1 is down
So the moral of the story: always check /var/log/messages if encounter an
unusual error. And please note that after every failed vinum create, you must reset
your vinum configuration. See the next section for more detail on how to do that.
Vinum reset
One thing to remember is that you sometimes have to remove your existing vinum
configuration. If you encounter any errors during the create, you should clear out
the vinum devices. This is because each create is cumulative. If youi keep
doing creates, you will wind up with more and more devices. Which is not necesarily
what you want.
Please note that resetting the configuration does not delete your data
but you must do another create with your original configuration.
Here's how I cleared out the vinum configuration:
# vinum resetconfig
WARNING! This command will completely wipe out your vinum
configuration.
All data will be lost. If you really want to do this, enter
the text
NO FUTURE
Enter text -> NO FUTURE
Vinum configuration obliterated
Don't use resetconfig unless you want to lose everything. Instead, use rm.
Mounting the vinum volume
This too is quite easy.
# mount /dev/vinum/test /mnt
There you go! Welcome to the vinum world! And if I typed mount,
here is what I found:
/dev/vinum/test on /mnt (local, writes: sync 2 async 1)
After all that, here is a set of successful logs from the above:
Aug 9 19:47:53 fred /kernel: vinum: drive d1 is up
Aug 9 19:47:53 fred /kernel: vinum: drive d2 is up
Aug 9 19:47:53 fred /kernel: vinum: test.p0.s0 is up
Aug 9 19:47:53 fred /kernel: vinum: test.p0.s1 is up
Aug 9 19:47:53 fred /kernel: vinum: test.p0 is up
Aug 9 19:47:53 fred /kernel: vinum: test is up
Aug 9 19:47:53 fred /kernel: vinum: test.p0 is up
Making it all automagical
This section shows how I made sure the vinum volume was always mounted when
the system started up.
First, I found the vinum options which are recognized by /etc/rc.conf.
# grep vinum /etc/defaults/rc.conf
start_vinum="" # set to YES to start vinum
Then I added the following to /etc/rc.conf.
start_vinum="YES" # set to YES to start vinum
vinum_drives="/dev/da1e /dev/da2e"
# put in names of disks containing vinum drives
# to start vinum on only specific disks.
Here's what I found in dmesg:
# dmesg | grep vinum
vinum: loaded
vinum: reading configuration from /dev/da2e
vinum: updating configuration from /dev/da1e
