zobie's blog I create software, I like music, and I'm mildly(?) OCD.

14Jul/09

Installing git on Ubuntu 9.04 (Jaunty)

Last night I was working on a new ubuntu 9.04 server on ec2 (it was ami-0d729464 from http://alestic.com if you're interested). Installing git via aptitude would have given me an older version so here's what I did.

$ sudo apt-get build-dep git-core git-doc libssl-dev
$ wget http://kernel.org/pub/software/scm/git/git-1.6.3.3.tar.gz
$ tar -xzf git-1.6.3.3.tar.gz
$ cd git-1.6.3.3/
$ ./configure
$ make all doc
$ sudo make install install-doc
$ git --version
git version 1.6.3.3
Tagged as: , , Comments Off
20Apr/09

Using git to avoid problems with TFS

For the past few months I've been using Team Foundation Server (TFS) at work. I'm certainly not a TFS expert; I probably don't even quality as a power-user. But I've used TFS enough to have found a handful of things that I like about it. Revision control is not among those things.

As a software version control system, I dislike TFS intensely.

In the short time I've been using TFS I've had several problems with code that was merged incorrectly. I've seen problems where TFS silently allowed older versions of code to overwrite newer versions. I could probably fill an entire blog post airing grievances with TFS but I thought it would be more interesting to describe how I use git on top of TFS to solve some of these problems.

First, to use git to track a TFS repository it is really important that all your source code be on a Fat32 partition. TFS locks files and NTFS respects that lock. Fat32 will track the lock but doesn't enforce it. This allows git to modify files (change to different versions of files) without necessarily having those files checked out in TFS.

Using TFS I checked out all my code into s:\src. I then created a new git repository in that same directory and added everything into the git repository.

For working I maintain at least two branches. My master branch always matches TFS. When I need the latest code from TFS I switch to the git master branch, pull from TFS then commit all changes into git. My working branch contains my current code changes. I also have one branch dev that contains a single commit consisting of all my debug code that should never be checked in to TFS.

When I'm ready to start coding I get the latest code from TFS and commit those changes into git's master branch. I create a new git branch, working. I cherry pick my development code from dev into working. Then I do all my coding on that branch. When I need to get code from TFS I can swtich to master, update from TFS, check that code in to git then either merge or rebase the changes back into working.

Once all of my changes in working are complete I need to merge the changes back into master so that I can commit them to TFS. I can't do a straight merge becuase my cherry-picked dev code would be included. So I have two ways of doing this:

  1. cherry-pick changes from working, applying them to master
  2. backout the development code (using git rebase -i) then merge changes back into master

After going through one of these two options I end up back on master with all of my code changes. I then commit the changes to TFS. Once that is done I delete working and recreate it from master next time I need it.

Working like this has been great for me. If there are conflicts when merging my code changes, git takes care of it. This way I can almost always avoid having to let TFS merge anything.

This is my general way of working but you can easily see how to apply these same principles when you want to work on multiple different changes using multiple different branches in git.

One thing to note: When you're working like this git's history isn't great. This isn't like git-svn where you get a seperate git revision for every svn revision. For me, using git with TFS isn't about being able to track my changes over time. I just want to make sure that my changes aren't lost and I don't want to clobber anyone else's changes.

14Jan/09

What’s so great about git?

Over the last few days, I have been asked several times why I like git. Many explanations and detailed comparisons exist that discuss how git compares to other version control systems. I'm not going to try and duplicate any of that work. These are just some thoughts about features in git that allow me to work more productively.

I didn't immediately grok git because it is fundamentally different from subversion. Subversion tracks the history of files whereas Git tracks the history of content. That difference may seem minor, but the shift brings a lot of flexibility. You can expect a small learning curve, but don't give up! If git works as well for you as it does for me, you will not want to go back to subversion.

So, what are my favorite things about git?

  • Branching: Coming from a CVS/SVN background, I didn't think git's "cheap branching" would matter to me. I am not exaggerating when I say that it has dramatically changed the way I work. Because branching and merging are so easy, I create a new branch for each feature or bug I'm working on. I am never tempted to check in broken code just because I want a checkpoint.
  • The index: It is really nice to be able to just check in a portion of the changes in my code rather than having to commit the entire file.
  • git stash: If I'm in the middle of some big change and need to fix a small bug, I can hide my outstanding changes, make the fix and restore my changes without a checkin.
  • git rebase: Sometimes it is really handy to be able to rewrite history.
  • Git is a DVCS: My work isn't crippled when I am not connected to the network. I can check code in and out. I can branch and merge. I have a full history of the repository. Awesome!
  • Very flexible workflow: With git I can push/pull to a central server like CVS or SVN. Alternatively, changes can bypass the server and be sent directly between working repositories.
  • Git is fast: Neither CVS nor SVN come close to git's speed.

Is git perfect? No. Off the top of my head, here are some things that I would really like to have in git:

  • git checkout -i: Sometimes I want to rollback a few changes in a file. I imagine this would work very similarly to 'git add -i'.
  • git stash -i: Sometimes I only want to stash some of my changes. This too would work like 'git add -i'.
  • Local tags: Local branches are great; local tagging would be too.
  • Track empty directories: I understand why this doesn't currently happen, but it can be obnoxious.
  • Comments on branches: It is easy to make a lot of branches but unless your branch names are super descriptive (and therefore super long), it is also easy to forget what each of them is for.
  • Better/more documentation: In general I am able to find what I'm looking for, but I still don't know what the "fetch-all" in git svn fetch --fetch-all actually does.
  • Windows support: Not that I've actually tried it, I've just heard that it doesn't work well.
  • GUI options: Most of the time I work in the cli but sometimes a gui is more convienent. I've heard multiple people ask for a tool that would allow them to branch and tag without checking out code. gitk and gitx just don't offer all the flexibility that some other tools do (e.g. WinCVS, SmartSVN, TortoiseSVN, etc.).

Why do you like (or hate) git?

14Dec/08

Managing WordPress Updates with Git

WordPress 2.7 was recently released so I started the process of updating the handful of WP instillations that I manage. Git has made this process a breeze! Here's how I do it.

  • To start a new project, use subversion to download the latest stable release of WordPress
  • Add that entire directory, including the .svn folders, to git. To keep subversion happy, make sure that all of the svn tmp directories are in git (use an empty .gitignore file to force git to add empty directories)
  • Before you make any changes, create a new branch in git called wordpress_base. You should never do anything with this branch other than update the WordPress code from subversion.
  • Switch back to master and change whatever needs changing (e.g. add themes, plugins, etc.). To make upgrades go as smoothly as possible you shouldn't modify the base WordPress files more than you absolutely have to but, adding themes and plugins should be no problem.
  • When everything is tested and ready to deploy, commit everything to git and push to the webserver

The command line will look something like this:

$ cd ~/Downloads
$ svn co http://svn.automattic.com/wordpress/branches/2.6/ new_blog
$ cd new_blog
$ git init
$ git add .
$ git commit -m "Import WordPress 2.6"
$ git branch wordpress_base
## Get the site completely setup, commit everything to git and
## push everything up to the server.

When you're ready to update WordPress:

  • Checkout the wordpress_base branch
  • Either 'svn up' or 'svn switch' to get the newest code
  • Check all of the changes in to git
  • Merge wordpress_base back in to master (I generally rebase then merge)
  • After everything in git has been committed, push, pull or do whatever fits your process to get the updated code from your development machine to the server
  • Don't forget to upgrade your blog database after the new code has been deployed by going to: http://myblog.com/wp-admin/upgrade.php
$ git checkout wordpress_base
$ svn switch http://svn.automattic.com/wordpress/branches/2.7
$ git add . && git ls-files --deleted | xargs git rm
$ git commit -m "Updated WordPress to version 2.7"
$ git checkout -b integration
$ git rebase master
$ git checkout master
$ git merge integration
$ git branch -d integration
## Make sure all conflicts have been resolved, test the new site and
## update plugins. Commit changes to git and push them to the server.

Tagged as: , , 1 Comment
2Dec/08

Migrating from svn to git

About a year ago I decided that it was time to try git. As a long-time user of SVN, CVS and even VSS *shudder*, I wanted to see what all the fuss was about. At first I didn't get it; having never used a DVCS before I just didn't grok the concept but I keep with it and one day it clicked. After a few weeks I was completely hooked and have since migrated most of my personal repositories from SVN (I haven't tried it under Windows so I haven't migrated my VisualStudio projects yet).

Not long after my conversion, we decided to start using git for a few projects at work. Since then we've been using both subversion and git on a self-hosted server. As anyone who gets git can tell you, after using a DVCS subversion can feel very limiting so we recently decided to migrate everything from subversion to git. We decided to host all of our repositories on github.

This is a long way to say that, over the past year I've migrated ~20 subversion repositories, of varying complexity, over to git. The process is not terribly difficult but there are multiple steps that you have to remember. I've kept a list of each step and thought that it might be useful to someone else (or perhaps myself when I need to do this again in a few months 😉 ).

To migrate the subversion repository my_great_app to git first you'll create an empty git repository. Note that this just does the setup, no import happens at this point.

$ git svn init -s svn://svn.zobie.com/svn/my_great_app my_great_app
$ cd my_great_app

Now, if you want your new git history to look pretty, you need to create a text file that maps subversion users to git users. For this example I'll name the file ~/users.text; it should look something like this (svn user on the left, git user on the right):

(no author) = Unknown Author <unknown@zobie.com>
nate = Nate Zobrist <zobie@zobie.com>

Then you tell git about your authors file and start the import:

$ git config svn.authorsfile ~/users.text
$ git svn fetch

Honestly, I don't really understand the "fetch-all" option but on one of the larger repositories that I migrated, the initial import didn't seem to be complete. I started completely over and when I ran "git svn fetch" I included the "--fetch-all" option. This significantly increased the amount of time that the import took but it seemed to do the trick.

$ git svn fetch --fetch-all

At this point your entire subversion repository has been imported into git and we want to create a tag in git that corresponds to each of the tags in subversion:

$ git branch -r | sed -rne 's, *tags/([^@]+)$,\1,p' | while read tag; do echo "git tag $tag 'tags/${tag}^'; git branch -r -d tags/$tag"; done | sh

Then, for each svn branch, we create a branch in git:

$ git branch -r | grep -v tags | sed -rne 's, *([^@]+)$,\1,p' | while read branch; do echo "git branch $branch $branch"; done | sh

These two commands will import every tag and branch from subversion. Sometimes git finds branches that you didn't know existed. Just delete whatever branches and tags that you don't want persisted.

Now that the repository is setup like we want, we do some housework to shrink and compact the repository.

$ git repack -d -f -a --depth=50 --window=100

Then we remove the meta-data that was used by git-svn:

$ git config --remove-section svn-remote.svn
$ git config --remove-section svn
$ rm -r .git/svn

At this point you have successfully migrated your subversion repository into git! If you aren't going to push this repository to a centrally hosted server you can stop at this point and enjoy using your new git repository.

If you want to keep a copy of your repository on a self-hosted server then create a bare clone and copy it up. Then configure your new server as "origin." You'll be happiest if you make sure that the master branch is checked out before cloning the repository.

$ git checkout master
$ cd ..
$ git clone --bare my_great_app my_great_app.git
$ scp -r my_great_app.git zobie@git.zobie.com:/git/.
$ rm -rf my_great_app.git
$ cd my_great_app
$ git remote add origin ssh://git.zobie.com/git/my_great_app.git

Alternatively, if you want to push the repository to github, use the the clone url that they provide:

$ git remote add origin git@github.com:zobie/my_great_app.git
$ git push --all && git push --tags

The last thing that you need to do is to point your "master" branch to "origin/master". There is probably a way to do this from the command line but I don't know it. I can't give exact instructions for modifying the config but the process isn't difficult. Open .git/config and point your master branch to origin; it should look something like this:

[branch "master"]
	remote = origin
	merge = refs/heads/master

If you have other local branches that should track their remote counterpart, you will need to make a similar change for each.

You can test that everything is working:

$ git pull
$ git push

Assuming you don't get any errors, you're done!

Update 2009-01-13: Corrected an error in the sed regex. The '1' should be escaped.