|
Things look quiet here. But I've been doing a lot of blogging at
dan.langille.org because I prefer WordPress now.
Not all my posts there are FreeBSD related.
I am in the midst of migrating The FreeBSD Diary over to WordPress
(and you can read about that here).
Once the migration is completed, I'll move the FreeBSD posts into the
new FreeBSD Diary website.
|
|
|
|
|
Creating your own cvsupd server
17 December 2000
|
|
|
With the upcoming work I'm going to be doing on FreshPorts, I thought it best to start up a cvsup server
for the source code. This server will be used to share the data amongst the other
FreshPorts developers. This article starts that process. I first wrote about cvsupd
on 6 August 1999. That article is still around, but now
that I'm installing a cvsupd server for my own needs, instead of creating a mirror, I found my existing notes a bit lacking.
Hence, this article.
I used John Polstra's CVSUP
FAQ for this exercise. Of note is the section on setting up a test server.
It's what I used as the basis for this article. |
|
|
The install
|
Well, seeing as I had all the port skeletons installed, I
just had to do the following:
cd /usr/ports/net/cvsupd-bin
make install
NOTE: in more recent versions of FreeBSD, cvsupd can be found in net/cvsup-without-gui.
Yet cvsupd is started by /usr/local/etc/rc.d/cvsupd which is installed by net/cvsup-mirror
cvsupd_enable="YES" in /etc/rc.conf
cvsupd_flags="-c sup:/home/repositories/sup"
Turn off your old cvsupd to avoid accidents.
|
|
|
Setting up a test server
|
In this section, I'm going to show you the bare minimum required to get
the server running so you can talk to it from a client.
This is the main directory which cvsupd uses and its where the repositories
will reside. In this example, we'll use /usr/local/etc/cvsup (which is
the default base directory for cvsupd). See the -b option.
(i.e. mkdir /usr/local/etc/cvsupd/sup). See the -c option.
- Create the
test collection directory:
This directory will contain the test collection
(i.e. mkdir /usr/local/etc/cvsupd/sup/test)
The releases file identifies the releases which are associated with this
collection.
Create /usr/local/etc/cvsupd/sup/test/releases to contain the following:
cvs list=list.cvs prefix=/home/repositories
This defines the prefix. Our repository will exist at /home/repositories
- Create the
list.cvs file:
This file contains the rules for use when processing the cvs release. In this
case, it contains the freshports-db collection. Here's what's in that
file:
upgrade freshports-db
This file should be in the directory /usr/local/etc/cvsupd/sup/test/.
This is your repository and it is relative to the prefix supplied in the releases
file. In our example, the repository will exist at /home/repositories/freshports-db.
It is the contents of this subtree which will be transferred.
|
|
|
Start the server
|
Here's where we start the server with a very simple situation:
/usr/local/sbin/cvsupd
We have not specified a base directory. cvsupd will use the default
directory, which we also used to create our base directory. We
have not specified the -e option, which means cvsupd will run from the command
line and not go into the background. Here is what it looks like when cvsupd
starts:
# /usr/local/sbin/cvsupd
2000.12.17 16:54:40 NZDT [75872]: CVSup server started
2000.12.17 16:54:40 NZDT [75872]: Software version: REL_16_1
2000.12.17 16:54:40 NZDT [75872]: Protocol version: 16.1
2000.12.17 16:54:40 NZDT [75872]: Ready to service requests
This method is fine for testing, but I actually run cvsupd as the nobody and
ensure that everything it serves is readable by everyone. cvsupd does not
create or write files, so from a security point of view, the risk of cvsupd being
tricked into damaging your system is actually quite low.
I start cvsupd automatically at system startup by creating /usr/local/etc/rc.d/cvsupd.sh,
which is chmod 770:
#!/bin/sh
[ -x /usr/local/sbin/cvsupd ] && \
su -m nobody -c "/usr/local/sbin/cvsupd -e -C 8 -l @daemon" && \
echo -n ' cvsupd'
|
|
|
Setting up a test client
|
We will now set up the test client to use our test server. We are
doing this on the same box as cvsupd is running. Create the directory ~/cvs-test.
In this directory, create supfile and place the following
in it:
*default host=localhost
*default base=.
*default release=cvs
*default delete use-rel-suffix
test
This is our cvsup configuration file. We have specified that we will be
connecting to the cvsup server on localhost (i.e. this box) and that we
will be obtaining the cvs release. From that release, we want the test
collection.
To run cvsup on the client and pull down the collection, issue the following command:
cvsup supfile
Here is a successful connection and cvsup:
$ cvsup supfile
Connected to localhost
Updating collection test/cvs
Mkdir freshports-db
Create freshports-db/freshports.db.sql,v
Create freshports-db/permissions.txt,v
SetAttrs freshports-db
Finished successfully
In this case, it's brought down the files associated with my repository. |
|
|
|
In this example, I'm going to add a very simple collection, test2,
which contains only contrib/top.
This isn't really a practical example, but it might help to illustrate how the
various components of cvsup fit together.I'll create the collection directory:
mkdir /usr/local/etc/cvsup/sup/test2
In that directory, I'll specify the releases associated with this collection by
creating releases which contains the following (all on one line):
cvs list=list.cvs prefix=/home/repositories/FreeBSD/ncvs/src/contrib/
This indicates that the repository actually exists at /home/repositories/FreeBSD/ncvs/src/contrib/.
Then I create the list.cvs file, the same directory as releases, to
contain this:
upgrade test2
I can use the same supfile as in the previous example, but I change test
to top.
Now when I cvsup, I get this:
$ cvsup supfile
Connected to localhost
Updating collection test2/cvs
Checkout top/ADVERTISEMENT
Checkout top/Changes
Checkout top/Configure
Checkout top/DISCLAIMER
Checkout top/FAQ
Checkout top/INSTALL
Checkout top/Make.desc.X
Checkout top/Makefile.X
Checkout top/Porting
Checkout top/README
Checkout top/boolean.h
Checkout top/commands.c
Checkout top/display.c
Checkout top/display.h
Checkout top/getans
Checkout top/getopt.c
Checkout top/install
Checkout top/layout.h
Checkout top/loadavg.h
Checkout top/m-template
Checkout top/machine.h
Checkout top/metatop
Checkout top/os.h
Checkout top/patchlevel.h
Checkout top/prime.c
Checkout top/screen.c
Checkout top/screen.h
Checkout top/sigconv.awk
Checkout top/top.X
Checkout top/top.c
Checkout top/top.h
Checkout top/top.local.H
Checkout top/username.c
Checkout top/utils.c
Checkout top/utils.h
Checkout top/version.c
Finished successfully
Compare the above list of files you can find in the CVS repository for contrib/top.
And if I check my directory, I see this:
$ ls
sup supfile top
And if I look in top, I see the same files as are listed above. |
|
|
Adding another bit to test2
|
Now lets try another example. Let's add contrib/awk to
the test2 collection.I'll assumed you have already created the example
from the previous section. To add awk to the
collection, modify /usr/local/etc/cvsup/sup/test2/list.cvs to contain this:
upgrade top
upgrade awk
Now run cvsup again. You should see this:
$ cvsup supfile
Connected to localhost
Updating collection test2/cvs
Checkout awk/ACKNOWLEDGMENT
Checkout awk/COPYING
Checkout awk/ChangeLog
[snip]
Checkout awk/test/subslash.awk
Checkout awk/test/subslash.ok
Checkout awk/test/zeroflag.awk
Checkout awk/test/zeroflag.ok
Checkout awk/version.c
Finished successfully
and your main directory should contain this:
$ ls
awk sup supfile top
|
|
|
Putting things in a different place
|
The supfile example contains a base specifiation of base=.
which puts everything from cvsup into the current directory. That's not the ideal
situation, but it's what we used for our testing. Now let's change that. Let's
put everything into /home/dan/mysupfiles. The first step is to modify
our supfile from the previous examples and change base to look like this:
*default base=/home/dan/mysupfiles
Don't forget to create the directory before you run cvsup:
mkdir /home/dan/mysupfiles
Then we cvsup again:
$ cvsup supfile
Connected to localhost
Updating collection test2/cvs
Checkout awk/ACKNOWLEDGMENT
Checkout awk/COPYING
Checkout awk/ChangeLog
[snip]
Checkout top/top.h
Checkout top/top.local.H
Checkout top/username.c
Checkout top/utils.c
Checkout top/utils.h
Checkout top/version.c
Finished successfully
Now if I check my directory, I'll find this:
$ ls /home/dan/mysupfiles/
awk sup top
NOTE: If you get this error:
$ cvsup supfile
Nonexistent base directory "~/mysupfiles" for collection "test2"
Then you didn't create ~/mysupfiles before running cvsup. |
|
|
|
This section will concentrate on restricting access to your cvsup server
by authenticating the connections. Authentication is controlled by the existence of
the file cvsupd.access which must appear in the base directory (in the example
above, the base directory is /usr/local/etc/cvsup). If this file does not
exist, no authentication takes place.The file cvsupd.passwd (again, in the
base directory) contains the authentication rules. Here's the contents I was testing
with:
cvsup.example.org:mysecretkey
dan@develop.example.org:$md5$c2e5a85c8042ce9f8c71bcc0f52876c8::
mysecretkey can be any key you choose. The second line was created using cvpasswd:
$ cvpasswd dan@develop.example.org cvsup.example.org
Enter password:
Enter same password again:
Send this line to the server administrator at cvsup.example.org:
----------------------------------------------------------------
dan@develop.example.org:$md5$c2e5a85c8042ce9f8c71bcc0f52876c8::
----------------------------------------------------------------
Be sure to send it using a secure channel!
Add this line to your file "$HOME/.cvsup/auth", replacing "XXX"
with the password you typed in:
----------------------------------------------------------------
cvsup.example.org:dan@develop.example.org:XXX:
----------------------------------------------------------------
Make sure the file is readable and writable only by you!
The password I entered was "secret". So this is what I placed in ~/.cvsup/auth
on my client machine:
cvsup.example.org:dan@develop.example.org:secret:
That should be enough to authenticate everyone. |
|
|
Access Control
|
Perhaps you don't want everyone accessing your cvsup server. I know I don't. So I've added some
rules to my packet filter to restrict access. People scanning the box can't
even tell there is a cvsup server there. But multiple layers of protection can be a good thing. That's
why I added more. If you read the man page for cvsupd, you'll see that that cvsupd.access
is your friend. I added a few rules to this file, which normally resides under
/usr/local/etc/cvsup, which permitted certain trusted host unlimited access to my server.
#
# allow anything from our local network
+10.0.0.0/24
# allow connections from local host
+127.0.0.1/32
# everything else must authenticate
*0.0.0.0/0
Using the above scenario, I didn't have to do anything special for boxes on my 10.0.0.* network
and the cvsup server could access itself. Everyone else, including my remote boxes, must
authenticate using the methods described in the previous section.
|
|
|
Secrecy
|
From man cvsupd:
If secrecy is desired then the connection can be tunneled
through ssh.
I tried to get this to work, but I failed. If anyone has a practical example,
please add it to the comments section. Thanks.
#!/bin/sh
ssh -f -L 32884:example.org:5999 dan@example.org sleep 30
cvsup -h localhost -p 32884 supfile
|
|
|
Common problems and solutions
|
If you encounter this message:
$ cvsup supfile
Connected to localhost
Server message: Collection "test" release "cvs" is not available here
Skipping collection test/cvs
Finished successfully
...it means the information contained in the releases file is
incorrect. In my case, I had put my data at /usr/repositories instead of /home/repositories
as specified in releases. Once I moved repositories to /home,
all was well.
If you encounter this message:
$ cvsup supfile
Connected to localhost
Server message: Unknown collection "test"
Skipping collection test/cvs
Finished successfully
...it means the collection specified in your client's supfile does not exist on the
server. Once of them is wrong. In our example, it means /usr/local/etc/cvsupd/sup/test
does not exist.
This message:
$ cvsup supfile
Cannot connect to localhost: Connection refused
Will retry at 16:53:45
...means that cvsupd isn't accepting connections. Make sure it's
running. Make sure you are allowed to connect. Check your authentication.
If you get this:
$ cvsup supfile
Connected to cvsup.freshports.org
Server error: Authentication failed
...then you have the wrong authentication information at either the client or the
server. Check both and try again. |
