If you work in a development environment, theres a good chance you are using Subversion as your code repository of choice. If thats the case, the usual suggestion for backing up is to dump the repositories onto a DVD or external drive to be stored offsite. We have been doing this for a while and have found the process painfull (to say the least!). If you run subversion and don’t have your data backed up frequently offsite, you might find yourself pushing this button sooner or later!
Near the end of last year I started looking at offsite backup options that didn’t require user intervention and was very excited to discover the svnsync command. The benefit of svnsync is that only new revisions are mirrored and not the full repository each time. This is absolutely critical if you have a repository that is quite active. Needless to say, I decided to forge ahead and try my hand at implementing automated scripts to take care of backing up our repositories online utilising the svnsync tool. As a reference I have posted the setup process here.
Its important to note that this guide assumes you are working in a Windows environment and that you have access to a server offsite. I have referenced a few articles and other blog posts I discovered along the way to help you if you are working in a Linux environment.
Step 1 – Setup Subversion on your remote server
Create a Windows user account on your remote server which you will use to remotely access the backup repository from your main Subversion server. Take note of the account name and password you use. Once you have created the account, install Visual SVN on the server where you want to host your mirrored repositories. Ensure you select Windows Authentication on the security dialog during the installation process. Once completed, ensure that Subversion is running correctly on the remote machine by opening the VisualSVN manager and clicking on the repository address displayed. Now ensure you can access the repository from your host Subversion server. If your backup server’s name is not addressable from your host server, use the remote servers IP address or simply add an entry to your DNS server or Windows host file. If you opted for setting up a DNS entry, you should be able to ping your backup server using the server’s name. Try access the repository again. When prompted for username and password use the credentials setup for the user account you created.
Step 2 – Configure permissions
Before setting up the repositories etc. we need to define which users have access writes to the backup repository. To configure this, open VisualSVN manager on the remote server and right click on the Repositories folder, choose Properties from the drop down menu. Revoke all access for the BUILTIN\Users role and then add the user account you setup in Step 1. Ensure this user has full Read/Write access.
Step 3 – Create the destination repository
Now that you have full configured the Subversion server hosted on your remote server we can start the process of setting up the synchronisation process. To do this we need to ensure that we have a destination repository to mirror your existing repository to (if you have more than one you need to create a destination repository for each repository you want to mirror). To keep things simple, I gave the destination repository the same name as the source repository. Take note that any repository you create on the destination server should be empty (i.e. do not tick the “Create default structure” checkbox when creating the repository.
Step 4 – Configure the repository
The next step involves setting the Pre-revision Property Change Hook. This is an important step. Right click on the repository you created on the destination server and select All Tasks > Manage Hooks. Click on the “Pre-revision property change hook” entry and click Edit. Enter a few blank lines and click OK and Apply.
Step 5 – Configure SSL
We need to configure the client server to accept the SSL certificate generated by the VisualSVN installer. If you wish to use a properly signed certificate or already have one, follow this guide and ignore the rest of this step. If you want to continue using the auto generated certificate, follow Mark Wilson’s guide on how to trust the default certificate.
Step 6 – Initialise your repositories for synchronisation
Before you can synchronise your repository, you need to initialise it. To do this, you need to run the following command on the host server (note that you need to replace the keys in CAPS to the relevant object names):
svnsync init PATH_TO_REMOTE_REPO PATH_TO_LOCAL_REPO –sync-username REMOTE_USERNAME –sync-password REMOTE_PASSWORD –source-username HOST_USERNAME –source-password HOST_PASSWORD
Step 7 – Initialise remote repositories from a previous backup
Only run through this step if you have a relatively large repository and don’t want to have to mirror it (the sync process is quite slow) from revision 0 all the way to revision xxxx. If you are running through these steps for a brand new repository you want to have mirrored, ignore this step. Also, please take note that if you are using Powershell to execute all these scripts “>” is equiv to | Out-File -encoding Unicode (thanks Keith). If you don’t be careful, you might end up with the Malformed dumpfile header error. To be safe, use the command line interface.
Dump your existing repository on your host machine by running the following script:
svnadmin dump “FILE_PATH_TO_REPO” > “REPO_NAME.db”
Once the repository dump has completed, upload it to your backup server and then run the following script on the backup/mirros server:
svnadmin load “FILE_PATH_TO_BACKUP_REPO” < "REPO_NAME.db"
Now, the next step is critical. You need to update the last-merged-rev property on the remote repository to the existing revision number of the repository (you can get this information by running “svn info REPO_PATH”). To do this run the following script:
svn propset svn:sync-last-merged-rev –revprop -r0 REV_NUMBER “PATH_TO_REMOTE_REPO”.
Step 8 – Synchronise!
Basically you are done, you simply need to run the following script on a frequent basis (best to setup as a scheduled task in Windows):
svnsync sync PATH_TO_REMOTE_REPO –sync-username REMOTE_USERNAME –sync-REMOTE_PASSWORD –source-username HOST_USERNAME –source-password HOST_PASSWORD
Hope you found this useful. I might follow this post up with another blog entry on steps I took to setup an automated script to email me when a repository on the host machine is missing its mirrored counterpart. This is really helpful to detect cases where a repository was setup locally but not configured for synchronisation, furthermore the ability to automatically generate the relevant scripts is quite useful