The FreeBSD Diary

I want to set up some jails. They will each be very similar. They will each be used to test a slightly different configuration of Bacula. My tool of choice is ezjail, available in the ports tree.

With ezjail, I can:

• create a jail flavour, upon which the creation of other jails can be based
• centrally update the jail's ports tree

The above does not fully describe the neat things you can do with ezjail. Read below to discover more fun and interesting things.

DISCLAIMER: I installed ezjail several months ago. I am only now getting around to documenting and writing about it. I may have omitted some steps. If so, I apologize; please let me know.

While creating my jails I used the following references:

• Jail administration framework
• The Quick-N-Dirty Guide to ezjail in FreeBSD
• AppserverJailsHOWTO

First step installation:

cd /usr/ports/sysutils/ezjail
make install clean

Remember to add this to /etc/rc.conf:

ezjail_enable="YES"

cd /usr/local/etc
cp ezjail.conf.sample ezjail.conf

This command creates the base jail. The base jail is the one upon which all other jails will be based. Note that this does not create a jail, it creates a base jail, one of the foundation components of your ezjail configuration.

ezjail-admin update -ip

This assumes you have already done a build world and that it exists within the standard and default location: /usr/src.

After running this command you should see this:

# ls /usr/jails
basejail        flavours        newjail

It also installs a copy of the ports tree into the base jail, using portsnap. I encountered this error while trying to create a new basejail. Upgrading to the latest STABLE fixed this problem.

install -o root -g wheel -m 444 dir-tmpl /usr/jails/fulljail/usr/share/info/dir
install:No such file or directory

The host system needs to have some IP addresses that will be used exclusively by the jails. That is, one IP address per jail. For ease of use, and personal sanity, I've added the following entries to my private DNS server:

; jails for testing Bacula

mysql41 IN A 10.10.10.100
mysql50 IN A 10.10.10.101
mysql51 IN A 10.10.10.102
pg73    IN A 10.10.10.103
pg74    IN A 10.10.10.104
pg80    IN A 10.10.10.105
pg81    IN A 10.10.10.106
pg82    IN A 10.10.10.107
pg83    IN A 10.10.10.109
sqlite3 IN A 10.10.10.108

Corresponding to the above, here are the ifconfig aliases I added to /etc/rc.conf on the host system:

ifconfig_fxp0_alias0="inet 10.10.10.100/32"
ifconfig_fxp0_alias1="inet 10.10.10.101/32"
ifconfig_fxp0_alias2="inet 10.10.10.102/32"
ifconfig_fxp0_alias3="inet 10.10.10.103/32"
ifconfig_fxp0_alias4="inet 10.10.10.104/32"
ifconfig_fxp0_alias5="inet 10.10.10.105/32"
ifconfig_fxp0_alias6="inet 10.10.10.106/32"
ifconfig_fxp0_alias7="inet 10.10.10.107/32"
ifconfig_fxp0_alias8="inet 10.10.10.108/32"
ifconfig_fxp0_alias9="inet 10.10.10.109/32"

The following command creates the first jail. You can create additional jails by changing the hostname and the IP address. Each jail needs a unique IP address and hostname. Ideally, the IP address will correspond to the IP address you have selected.

# ezjail-admin create -f bacula mysql41.example.org 10.10.10.100
/usr/jails/mysql41.example.org/COPYRIGHT
/usr/jails/mysql41.example.org/basejail
/usr/jails/mysql41.example.org/bin
/usr/jails/mysql41.example.org/boot
/usr/jails/mysql41.example.org/dev
...
...
...
/usr/jails/mysql41.example.org/var/cfengine/inputs
/usr/jails/mysql41.example.org/var/cfengine/inputs/update.conf
/usr/jails/mysql41.example.org/var/cfengine/master
40797 blocks
Note: Shell scripts installed, flavourizing on jails first startup.
Warning: Some services already seem to be listening on all IP, (including 10.55.0.100)
  This may cause some confusion, here they are:
root     bacula-fd  1023  3  tcp4   *:9102                *:*
root     sshd       1010  4  tcp4   *:22                  *:*
root     master     995   11 tcp4   *:25                  *:*
root     syslogd    816   7  udp4   *:514                 *:*
[root@polo ~]# jls

Of note, you see the services running on the host which are listening on all IP addresses. This will be a problem for the jails. I will need to change the host services to listen only on the main IP address for this host. Sometimes you might want a host service to listen on all IP addresses, but that's not what I want here.

The -f argument indicates that the jail should be created using the bacula flavour. This flavour was set up back in March, and I do not have my notes from that work. I will probably be creating more flavours and if I do, I will document them. But I do have a file list. Here is is the list of files from my bacula ezjail flavour:

[dan@polo /usr/jails/flavours/bacula]$ find .
.
./etc
./etc/mail
./etc/mail/aliases
./etc/mail/mailer.conf
./etc/ssh
./etc/ssh/ssh_host_rsa_key.pub
./etc/ssh/ssh_host_rsa_key
./etc/ssh/ssh_host_key.pub
./etc/ssh/ssh_host_key
./etc/ssh/ssh_host_dsa_key.pub
./etc/ssh/ssh_host_dsa_key
./etc/rc.conf
./etc/crontab
./etc/syslog.conf
./etc/make.conf
./etc/resolv.conf
./etc/periodic.conf
./usr
./usr/local
./usr/local/etc
./usr/local/etc/sudoers
./usr/home
./usr/home/dan
./usr/home/dan/.ssh
./usr/home/dan/.ssh/id_dsa
./usr/home/dan/.ssh/id_dsa.pub
./usr/home/dan/.ssh/id_dsa_no_passphrase
./usr/home/dan/.ssh/id_dsa_no_passphrase.pub
./usr/home/dan/.ssh/authorized_keys
./usr/home/dan/.ssh/known_hosts
./pkg
./pkg/joe.tbz
./pkg/libiconv.tbz
./pkg/gettext.tbz
./pkg/cvsup-without-gui.tbz
./pkg/cfengine-2.2.3_1.tbz
./pkg/db46-4.6.21.0.tbz
./pkg/aspell.tbz
./pkg/bash.tbz
./var
./var/cfengine
./var/cfengine/inputs
./var/cfengine/inputs/update.conf
./var/cfengine/master
./ezjail.flavour
[dan@polo /usr/jails/flavours/bacula]$ 

See Modifying other daemons in the Jails under FreeBSD 6 article for more information on how I've done this in the past.

Now that I have created my jail, it is time to start it:

#  /usr/local/etc/rc.d/ezjail.sh start
 ezjailConfiguring jails:.
Starting jails: mysql41.example.org.

I verified the jail was running:

# jls
   JID  IP Address    Hostname                Path
     1  10.10.10.100  mysql41.example.org     /usr/jails/mysql41.example.org

I was able to ssh to the newly started jail:

$ uptime
 1:19PM  up  1:33, 1 user, load averages: 0.00, 0.00, 0.00
$ hostname
mysql41.example.org
$ 

NOTE: this attempt failed. You may not want to try this. Read the whole section first, then see what actually succeeded.

After logging into the new jail, I noticed my shell was missing my favourite bash prompt. I decided to copy the required files into the flavour and refresh the jail.

cp ~/.bash_profile ~/.bashrc /usr/jails/flavours/bacula/usr/home/dan/

# /usr/local/etc/rc.d/ezjail.sh stop
 ezjailStopping jails: mysql41.example.org.
# ezjail-admin delete mysql41.example.org
# ezjail-admin create -x -f bacula mysql41.example.org 10.10.10.100
Warning: Some services already seem to be listening on all IP, (including 10.10.10.100)
  This may cause some confusion, here they are:
root     bacula-fd  1483  3  tcp4   *:9102                *:*
root     master     994   11 tcp4   *:25                  *:*
root     syslogd    815   7  udp4   *:514                 *:*

As you can see, I still have a few daemons listening on all addresses. I will handle that eventually. However, my goal of recreating the jail with my bash configuration files failed.

What I had to do was delete the jail with the -w option, to wipe the jail from the HDD. Then create it again:

ezjail-admin delete -w mysql41.example.org
ezjail-admin create -f bacula mysql41.example.org 10.10.10.100
/usr/local/etc/rc.d/ezjail.sh start

Now I will create another jail for testing MySQL 5.0:

[root@polo /usr/jails/flavours/bacula]# ezjail-admin create -f bacula mysql50.example.org 10.55.0.101
/usr/jails/mysql50.example.org/COPYRIGHT
/usr/jails/mysql50.example.org/basejail
/usr/jails/mysql50.example.org/bin
/usr/jails/mysql50.example.org/boot
/usr/jails/mysql50.example.org/dev
...
...
/usr/jails/mysql50.example.org/var/cfengine
/usr/jails/mysql50.example.org/var/cfengine/inputs
/usr/jails/mysql50.example.org/var/cfengine/inputs/update.conf
/usr/jails/mysql50.example.org/var/cfengine/master
/usr/jails/mysql50.example.org/var/ports
/usr/jails/mysql50.example.org/var/ports/packages
67260 blocks
Note: Shell scripts installed, flavourizing on jails first startup.
Warning: Some services already seem to be listening on IP 10.55.0.101
  This may cause some confusion, here they are:
root     ntpd       65252 8  udp4   10.55.0.101:123       *:*
Warning: Some services already seem to be listening on all IP, (including 10.55.0.101)
  This may cause some confusion, here they are:
root     ntpd       65252 4  udp4   *:123                 *:*
[root@polo /usr/jails/flavours/bacula]#  /usr/local/etc/rc.d/ezjail.sh      
Usage: /usr/local/etc/rc.d/ezjail.sh [fast|force|one](start|stop|restart|rcvar|startcrypto|stopcrypto)
[root@polo /usr/jails/flavours/bacula]#  /usr/local/etc/rc.d/ezjail.sh start
ezjailConfiguring jails:.
Starting jails: mysql50.example.org [mysql41.example.org already running 
                            (/var/run/jail_mysql41_unixathome_org.id exists)].
[root@polo /usr/jails/flavours/bacula]# 

And here is that jail:

$ hostname
mysql50.example.org
$ uname -a
FreeBSD mysql50.example.org 7.0-STABLE FreeBSD 7.0-STABLE #4: Fri Jul  
4 14:01:32 EDT 2008     dan@polo.example.org:/usr/obj/usr/src/sys/PHENOM  
amd64
$ bash
[dan@mysql50:/usr/home/dan] $ cd
[dan@mysql50:~] $ uptime
 6:03PM  up  6:17, 1 user, load averages: 0.07, 0.02, 0.02

You can start/stop individual jails.

#  /usr/local/etc/rc.d/ezjail.sh stop mysql50.example.org
Stopping jails: mysql50.example.org.
#  /usr/local/etc/rc.d/ezjail.sh start mysql50.example.org
Configuring jails:.
Starting jails: mysql50.example.org.
#

I've decided it is time to upgrade these jails. The host system is now running:

FreeBSD polo.example.org 7.0-STABLE FreeBSD 7.0-STABLE #5: Fri Aug 22 13:45:47 EDT 2008     dan@polo.example.org:/usr/obj/usr/src/sys/PHENOM  amd64

Whereas, the jails are:

FreeBSD mysql50.example.org 7.0-STABLE FreeBSD 7.0-STABLE #5: Fri Aug 22 13:45:47 EDT 2008    dan@polo.example.org:/usr/obj/usr/src/sys/PHENOM  amd64

Yes, they look identical. But to upgrade the ezjail after doing a build world in there, I did this:

That's my first cut of using ezjail. I'll be amending and adding to my flavours as I add more jails. I also plan to use cfengine to configure the jails, install/upgrade ports, and to undertake general maintenance. Watch for cfengine in an upcoming article.