zobie's blog I create software, I like music, and I'm mildly(?) OCD.

2Dec/08

Migrating from svn to git

About a year ago I decided that it was time to try git. As a long-time user of SVN, CVS and even VSS *shudder*, I wanted to see what all the fuss was about. At first I didn't get it; having never used a DVCS before I just didn't grok the concept but I keep with it and one day it clicked. After a few weeks I was completely hooked and have since migrated most of my personal repositories from SVN (I haven't tried it under Windows so I haven't migrated my VisualStudio projects yet).

Not long after my conversion, we decided to start using git for a few projects at work. Since then we've been using both subversion and git on a self-hosted server. As anyone who gets git can tell you, after using a DVCS subversion can feel very limiting so we recently decided to migrate everything from subversion to git. We decided to host all of our repositories on github.

This is a long way to say that, over the past year I've migrated ~20 subversion repositories, of varying complexity, over to git. The process is not terribly difficult but there are multiple steps that you have to remember. I've kept a list of each step and thought that it might be useful to someone else (or perhaps myself when I need to do this again in a few months ;) ).

To migrate the subversion repository my_great_app to git first you'll create an empty git repository. Note that this just does the setup, no import happens at this point.

$ git svn init -s svn://svn.zobie.com/svn/my_great_app my_great_app
$ cd my_great_app

Now, if you want your new git history to look pretty, you need to create a text file that maps subversion users to git users. For this example I'll name the file ~/users.text; it should look something like this (svn user on the left, git user on the right):

(no author) = Unknown Author <unknown@zobie.com>
nate = Nate Zobrist <zobie@zobie.com>

Then you tell git about your authors file and start the import:

$ git config svn.authorsfile ~/users.text
$ git svn fetch

Honestly, I don't really understand the "fetch-all" option but on one of the larger repositories that I migrated, the initial import didn't seem to be complete. I started completely over and when I ran "git svn fetch" I included the "--fetch-all" option. This significantly increased the amount of time that the import took but it seemed to do the trick.

$ git svn fetch --fetch-all

At this point your entire subversion repository has been imported into git and we want to create a tag in git that corresponds to each of the tags in subversion:

$ git branch -r | sed -rne 's, *tags/([^@]+)$,\1,p' | while read tag; do echo "git tag $tag 'tags/${tag}^'; git branch -r -d tags/$tag"; done | sh

Then, for each svn branch, we create a branch in git:

$ git branch -r | grep -v tags | sed -rne 's, *([^@]+)$,\1,p' | while read branch; do echo "git branch $branch $branch"; done | sh

These two commands will import every tag and branch from subversion. Sometimes git finds branches that you didn't know existed. Just delete whatever branches and tags that you don't want persisted.

Now that the repository is setup like we want, we do some housework to shrink and compact the repository.

$ git repack -d -f -a --depth=50 --window=100

Then we remove the meta-data that was used by git-svn:

$ git config --remove-section svn-remote.svn
$ git config --remove-section svn
$ rm -r .git/svn

At this point you have successfully migrated your subversion repository into git! If you aren't going to push this repository to a centrally hosted server you can stop at this point and enjoy using your new git repository.

If you want to keep a copy of your repository on a self-hosted server then create a bare clone and copy it up. Then configure your new server as "origin." You'll be happiest if you make sure that the master branch is checked out before cloning the repository.

$ git checkout master
$ cd ..
$ git clone --bare my_great_app my_great_app.git
$ scp -r my_great_app.git zobie@git.zobie.com:/git/.
$ rm -rf my_great_app.git
$ cd my_great_app
$ git remote add origin ssh://git.zobie.com/git/my_great_app.git

Alternatively, if you want to push the repository to github, use the the clone url that they provide:

$ git remote add origin git@github.com:zobie/my_great_app.git
$ git push --all && git push --tags

The last thing that you need to do is to point your "master" branch to "origin/master". There is probably a way to do this from the command line but I don't know it. I can't give exact instructions for modifying the config but the process isn't difficult. Open .git/config and point your master branch to origin; it should look something like this:

[branch "master"]
	remote = origin
	merge = refs/heads/master

If you have other local branches that should track their remote counterpart, you will need to make a similar change for each.

You can test that everything is working:

$ git pull
$ git push

Assuming you don't get any errors, you're done!

Update 2009-01-13: Corrected an error in the sed regex. The '1' should be escaped.

Comments (12) Trackbacks (0)
  1. Good stuff here, just what I needed. Thanks.

  2. When I try the command for importing tags I get the following error:

    fatal: Failed to resolve ‘tags/1^’ as a valid ref.
    error: remote branch ‘tags/1′ not found.

    A mystery to me…

  3. @alexis – I’m not sure what that means either but I wonder if the problem could be with the ‘^’ in the tag name. Does your subversion tag have special characters in it? I’m not sure how git would handle those, I’ve never tried.

  4. There is one tag “functional-1.0″ but that’s it. I’m running on Mac OS X. Maybe I’m using a different version of the command line tools than you? FWIW I’m using git 1.6.1, sed 4.1.5, and svn 1.5.1.

  5. @alexis I had to stare at it for a while but I think that I’ve found the problem. In the regular expression that sed uses the ’1′ needs to be escaped. The backslash must have been stripped out when the post was uploaded. I’ve corrected it above, thanks!

  6. That solved it. Thanks for the help.

    It’s bizarre to me that this requires so many steps. They discontinued the tool git-svnimport, but there doesn’t seem to be anything to replace it for clean one-shot imports.

  7. Thanks for the brilliant instructions!! This worked really smooth. The instructions on setting up the tags and branches for git is great. I haven’t seen any body else mention them. You hit the nail on the head with that one. :-)

  8. Nate, don’t take your blog down. I refer to this page every time I have to do a migration.

    I’ve had to do it a bunch lately as people I work with convert over to git.

    I’ve found the best way to convert people is to migrate svn to git, then delete the svn repo and all the backups.

  9. Awesome. Thanks for this! After trying several others approaches, this one worked perfectly.

    One small note: on OSX 10.5, sed does not have a -r switch. Instead, use the -E switch which accomplishes the same thing. e.g.:

    git branch -r | sed -Ene ‘s, *tags/([^@]+)$,\1,p’ | while read tag; do echo “git tag $tag ‘tags/${tag}^’; git branch -r -d tags/$tag”; done | sh

  10. I love this blog! Thank you so much! This was a life saver.

  11. Thanks! This helped a lot to migrate our project. The only extra command I have to use was:
    git update-server-info
    on the remote server. I don’t know what is does, but it helped.


Trackbacks are disabled.