May 112009

I wanted to change the default shell on my iPhone. Unfortunately chsh wasn’t available.

[10:40][root@iPhone:~]$ chsh
-bash: chsh: command not found

I next thought to modify /etc/passwd but, according to the file’s header, that’s not going to help.

[10:41][root@iPhone:~]$ cat /etc/passwd 
# 4.3BSD-compatable User Database
# Note that this file is not consulted for login.
# It only exisits for compatability with 4.3BSD utilities.
# This file is automatically re-written by various system utilities.
# Do not edit this file.  Changes will be lost.
nobody:*:-2:-2:Unprivileged User:/var/empty:/usr/bin/false
root:*:0:0:System Administrator:/var/root:/bin/sh
mobile:*:501:501:Mobile User:/var/mobile:/bin/sh
daemon:*:1:1:System Services:/var/root:/usr/bin/false
_sshd:*:75:75:sshd Privilege separation:/var/empty:/usr/bin/false
_unknown:*:99:99:Unknown User:/var/empty:/usr/bin/false

Finally I found /etc/master.passwd. I just modified root and mobile to use /bin/bash instead of /bin/sh and on my next login I was using bash!

[10:41][root@iPhone:~]$ cat /etc/master.passwd 
# User Database
# This file is the authoritative user database.
nobody:*:-2:-2::0:0:Unprivileged User:/var/empty:/usr/bin/false
root:5IS3K.2i/ciLw:0:0::0:0:System Administrator:/var/root:/bin/bash
mobile:NhbBgPU3IOnek:501:501::0:0:Mobile User:/var/mobile:/bin/bash
daemon:*:1:1::0:0:System Services:/var/root:/usr/bin/false
_sshd:*:75:75::0:0:sshd Privilege separation:/var/empty:/usr/bin/false
_unknown:*:99:99::0:0:Unknown User:/var/empty:/usr/bin/false
Apr 302009

I’ve found that the best way to deal with merging is to avoid it completely! Unfortunately that is rarely realistic. So, assuming you don’t want to take any radical measures to completely avoid merging in TFS, you should at least use the best tools available. My favorite merge tool is the freely available (and cross-platform) P4Merge.

Getting TFS to use P4Merge isn’t difficult but neither is it intuitive. For a merge operation P4Merge expects four files to exist:

  1. the original, base file
  2. file with conflicting change #1
  3. file with conflicting change #2
  4. final, merged file

Unfortunately TFS doesn’t create the merged file (#4) until after the merge tool is invoked. A simple batch script will solve the problem though. Save this as p4merge.bat.

COPY /Y NUL ""%4""
START /WAIT /D "C:\Program Files\Perforce\" p4merge.exe ""%1"" ""%2"" ""%3"" ""%4""

This script will create the merge file and invoke p4merge.exe.

Now you can configure TFS to use P4Merge by running this command from a Visual Studio command prompt: tf diff /configure

Visual Studio Command Prompt

That will bring up a dialog:

Configure User Tools

If an entry already exists for the Merge operation you can add it. Otherwise just modify the existing entry to point to the batch file we created:

Configure Tool

Note that you must set the command to be your batch file, not the executable.

And that’s it! Next time TFS launches a merge tool, it will use P4Merge.

Apr 202009

For the past few months I’ve been using Team Foundation Server (TFS) at work. I’m certainly not a TFS expert; I probably don’t even quality as a power-user. But I’ve used TFS enough to have found a handful of things that I like about it. Revision control is not among those things.

As a software version control system, I dislike TFS intensely.

In the short time I’ve been using TFS I’ve had several problems with code that was merged incorrectly. I’ve seen problems where TFS silently allowed older versions of code to overwrite newer versions. I could probably fill an entire blog post airing grievances with TFS but I thought it would be more interesting to describe how I use git on top of TFS to solve some of these problems.

First, to use git to track a TFS repository it is really important that all your source code be on a Fat32 partition. TFS locks files and NTFS respects that lock. Fat32 will track the lock but doesn’t enforce it. This allows git to modify files (change to different versions of files) without necessarily having those files checked out in TFS.

Using TFS I checked out all my code into s:\src. I then created a new git repository in that same directory and added everything into the git repository.

For working I maintain at least two branches. My master branch always matches TFS. When I need the latest code from TFS I switch to the git master branch, pull from TFS then commit all changes into git. My working branch contains my current code changes. I also have one branch dev that contains a single commit consisting of all my debug code that should never be checked in to TFS.

When I’m ready to start coding I get the latest code from TFS and commit those changes into git’s master branch. I create a new git branch, working. I cherry pick my development code from dev into working. Then I do all my coding on that branch. When I need to get code from TFS I can swtich to master, update from TFS, check that code in to git then either merge or rebase the changes back into working.

Once all of my changes in working are complete I need to merge the changes back into master so that I can commit them to TFS. I can’t do a straight merge becuase my cherry-picked dev code would be included. So I have two ways of doing this:

  1. cherry-pick changes from working, applying them to master
  2. backout the development code (using git rebase -i) then merge changes back into master

After going through one of these two options I end up back on master with all of my code changes. I then commit the changes to TFS. Once that is done I delete working and recreate it from master next time I need it.

Working like this has been great for me. If there are conflicts when merging my code changes, git takes care of it. This way I can almost always avoid having to let TFS merge anything.

This is my general way of working but you can easily see how to apply these same principles when you want to work on multiple different changes using multiple different branches in git.

One thing to note: When you’re working like this git’s history isn’t great. This isn’t like git-svn where you get a seperate git revision for every svn revision. For me, using git with TFS isn’t about being able to track my changes over time. I just want to make sure that my changes aren’t lost and I don’t want to clobber anyone else’s changes.

Strange Crash in OS X: securityd

 Technology  Comments Off on Strange Crash in OS X: securityd
Mar 232009

A few days ago my Mac started having problems. I would be in the middle of some task when it would suddenly refuse to launch any new applications. Whenever I tried to launch any app, it would bounce a few times in the dock then exit.

As far as I could tell, any apps that were running when I got into this problem state would continue working fine. The OS would never completely freeze but I noticed that my CPU started being monopolized by CrashReporter. I tried killing that process but it would just immediately relaunch and peg the CPU again. I looked inside /Library/Logs/CrashReporter/ and saw that a new crash log was being created about every three seconds. The crash logs were for many different applications but none of the stack traces was useful. I had trouble spotting a pattern to what might trigger the problem.

Once my box was in the bad state I tried to ssh in to see if I could gather any useful information. SSH would prompt me for a password but it always denied access saying that I had entered an invalid password.

The only way out of this state was to restart the machine. When I tried to reboot, OS X would successfully log out but then get stuck on a blue screen. I would see the indeterminate NSProgressIndicator for a few seconds then it would disappear for a few seconds then come back again. I was forced to power cycle the machine.

I finally noticed that when this problem occurred, the first crash log was always for securityd. /Library/Logs/CrashReporter/securityd_2009-03-23-204700_macpro.crash:

Process:         securityd [22]
Path:            /usr/sbin/securityd
Identifier:      securityd
Version:         ??? (???)
Code Type:       X86 (Native)
Parent Process:  launchd [1]
Date/Time:       2009-03-23 20:47:00.211 -0600
OS Version:      Mac OS X 10.5.6 (9G55)
Report Version:  6
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000001000000
Crashed Thread:  0

This information finally led me to the solution.

The problem was that securityd would crash then any app that needed to authenticate was unable to do so. One newsgroup noted that the problem could be temporarily solved by relaunching the process:

$ launchctl load /System/Library/LaunchDaemons/

After a bit more searching I found a permanent answer in a mailing list archive: Keychain access crashing on SecKeychainFindGenericPassword. The solution was incredibly simple (and completely unintuitive). I had to remove the file /var/db/CodeEquivalenceDatabase and reboot. That’s it!

The thread offers more details but basically, “that file [/var/db/CodeEquivalenceDatabase] has gotten corrupted and runs securityd into an endless memory-eating loop that (usually) ends up running your system out of memory and into the ground.”

Mar 112009

As I was recently looking for new employment I spent quite a bit of time deciding wether I might enjoy doing contract work full-time. I enjoy working on different projects and learning new things but there is one major roadblock to becoming a full-time contract developer. My personality doesn’t let me write software that is anything less than my best.

WARNING: Gross generalizations and simplifications below. I’m not trying to offend anyone, just describe my experiences.

Most of the contract work I’ve done has been for people who are not technically savvy. They come to me with a very vague idea of the software they want. They expect me to tell them how much it will cost before we’ve discussed specific requirements. When the requirements are incomplete or incorrect they expect that I’ll just fix it without additional cost to them.

Not all of my contract experience has been negative. In fact, most of it has worked out quite well. Usually both my client and myself are pleased with the software and the cost of building it. But I’ve had enough negative experiences to be careful when considering a new job.

Part of the problem is that it is nearly impossible for anyone to completely define the scope of a project. There is always some miscommunication or misunderstanding, there is always some unforseen problem.

“You want me to setup a blog for your company? No problem, I can get WordPress setup for you in an hour.”

“Wait, I didn’t realize that by ‘blog’ you meant store front application that can accept payments, handle accounts payable, accounts receivable and inventory tracking. That will take slightly more than an hour.”

That kind of situation actually isn’t bothersome to me. As a contractor it is part of my job to understand what you want before making a bid. If a potential client obviously doesn’t know what they want, I can either decline to bid on that job or I can adjust my bid to account for a large amount of unknown. I don’t love it, but that type of risk is manageable.

The part of contract work that I dislike is being forced to compromise quality. When I’m working on a fixed cost contract, it is in my best interest to deliver exactly what is specified, as quickly as possible. As long as my client is reasonably happy with the deliverable, I am going to get paid $20k regardless of whether it took two days, two weeks or two months to create. I don’t get additional money for clean code. I don’t get extra for having good test coverage.

When I complete a project more quickly than I had anticipated there is no problem. I can spend time verifying that the code is tight and that everything is working as expected. But if I am running behind schedule, it becomes more difficult to care about testing the code or fixing “little” bugs.

There may be a bug in the code where order totals aren’t calculated correctly, but what are the odds that my client will notice the bug before he signs off on the project? If he does find the problem and I correct it, will he think to test for that same bug in every release?

This is the dilemma that makes contract work difficult for me. If I see a bug in my code, I’m going to fix it. If I’m writing a tricky or important calculation (like calculating totals), I’m going to write a test. I need to have confidence that my code is doing what I expect. I’ve never shipped any software that didn’t have a list of known bugs but I have also never shipped any software in which I didn’t have a high level of confidence that it was working correctly.

For me, doing the bare minimum isn’t an option for two reasons:

  1. Quality is extremely important to me. I can’t just hack something together that meets the contract requirements. When I write software, I want to deliver my personal best.
  2. Most of the time, the fast/crappy way of implementing something simply doesn’t occur to me.

I understand a company’s need to understand cost before approving custom softare. But if you want me to do contract work, pay me on an hourly basis. I’ll give you a projected timeline for project completion.

With an hourly rate, you only pay me for the time I actually spend working. With an hourly rate I know that I won’t lose money just because I insist on high-quality code. We’ll both be happier in the long run.