Quick Recipe – setting up master-slave subversion servers with replication using svnsync and transparent proxy in Ubuntu

If you are having enough background, this article is a shortcut for you to set up a group of SVN servers with one master and one slave (you should be able to generalize it to multiple slaves with some minor touches). This group of SVN servers could be synchronized (or replicated) to allow multi-site deployment for network efficiency. However, you could always refer to this article for details on setting up svnsync and this article for setting up ssh login without password.

Assumed you are given:

  1. A master SVN server ([Master]) access via HTTP with a proper repository setup at /var/svn/repos. (You could refer to this article for setting up an SVN server.)
  2. A slave SVN server ([Slave]) accessed via HTTP (with an empty repository /var/svn/repos created as in this article).
  3. The credentials for accessing the master and slave SVN servers are identical.
  4. The master and slave SVN servers can access each other.

To set this up, follow the recipe:

  1. [Slave] cd /etc/apache2/mods-enabled; sudo ln -s ../mods-available/proxy.load; sudo ln -s ../mods-available/proxy_http.load
  2. [Slave] Add a SVNMasterURI directive under the <Location> directive in /etc/apache2/mods-available/dav_svn.conf (details)
  3. [Slave] sudo adduser svnsync
  4. [Slave] sudo adduser svnsync svn
  5. [Slave] sudo chown -R svnsync:svn /var/svn/repos
  6. [Slave] Create a script /var/svn/repos/hooks/pre-revprop-change (details)
  7. [Slave] sudo chmod +x /var/svn/repos/hooks/pre-revprop-change
  8. [Slave] sudo svnsync init –username svnsync file:///var/svn/repos http://<master hostname>/svn/repos
  9. [Slave] Download the script svn-sync-slave.sh and put under /var
  10. [Slave] Modify the first line of /var/svn-sync-slave.sh from #!/bin/sh to #!/bin/bash
  11. [Slave] sudo chown svnsync:svn /var/svn-sync-slave.sh; sudo chmod +x /var/svn-sync-slave.sh
  12. [Master] Create a script /var/svn/repos/hooks/post-commit (details)
  13. [Master] sudo chown www-data:svn /var/svn/repos/hooks/post-commit; sudo chmod g+rws /var/svn/repos/hooks/post-commit
  14. [Master] sudo -i
  15. [Master] su – www-data
  16. [Master] ssh-keygen -t rsa (give empty passphrase)
  17. [Master] ssh svnsync@<slave hostname> mkdir -p .ssh
  18. [Master] cat .ssh/id_rsa.pub | ssh svnsync@<slave hostname> ‘cat >> .ssh/authorized_keys’

To test this setup:

  1. Find another computer with svn client installed. ([Client])
  2. [Client] svn import –username <user> <path to local folder> http://<slave hostname>/svn/repos
  3. Check whether the imported folders and files are in both master and slave SVN servers’ repositories.
  4. [Client] Pick any one file imported.  Modify it and commit it to http://<slave hostname>/svn/repos.
  5. Check whether the modified file is in both master and slave SVN servers’ repositories.

Update on 2011-07-28:

There is something subtle that needs special consideration.

In fact, when I try to apply this procedures to an old and big svn repository, there is a problem.  I tried to start synchronizing with a new repository (you have to start from scratch anyway!) over the network.  I then tried to do a rough gauging and calculation exercise.  It needs to take 3 months to sync them all (yes, days and nights, continuously).

Luckily, we resolved it by using the file-based nature of svn.  We first make a sync in the same machine.  But note that you should use the target url as the master url, i.e. you should use IP / hostname to sync instead of localhost as svnsync remembers the url you used to init the sync process.

Then, after the sync process is done (which is much much much faster), zip the whole repository and upload to the target slave machine.  In the slave machine, you could expand and rename the repository as you like.  At this point, the svnsync becomes incremental.  So it would not take you much time to do a few revisions’ synchronization.

Details here:

[Slave] Add a SVNMasterURI directive under the <Location> directive in /etc/apache2/mods-available/dav_svn.conf

SVNMasterURI http://<master hostname>/svn/repos

[Slave] Create a script /var/svn/repos/hooks/pre-revprop-change

#!/bin/sh
USER="$3"

if [ "$USER" != "svnsync" ]; then
    echo >&2 "Only the svnsync user can change revprops"
    exit 1
fi

exit 0

[Master] Create a script /var/svn/repos/hooks/post-commit

#!/bin/sh

ssh -l svnsync <slave hostname> "/var/svn-sync-slave.sh -r /var/svn/repos"
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at WordPress.com.

Up ↑

%d bloggers like this: