I just returned from the 2010 edition of Apple's WWDC. The week before I took a week off from my iPhone and lived with an Android Nexus One. Both my personal experience with Android and the following week's announcements left a strong impression that, in the so-called "smartphone" space, we are really seeing the success of two very different mind-sets, cultures and philosophies. I'm not interested in a winner-takes-all kind of war. Instead I wanted to look at the differences between these two companies in the context of them co-existing and each having a happy, satisfied base of customers.

I was going to write about how capable the Nexus One seems to me, but how amateurish the UI appears. At times, the look and feel of the Android UI looks like its designers simply couldn't figure out how to fill the space with something interesting.

I was going to write about how the ubiquity of the four hardware buttons are both a blessing and curse. I understand that the designers were trying to provide a set of near-universal options in a consistent place. Unfortunately it was too easy to accidentally hit the "home" key while typing on the soft keyboard.

I was going to write about the moment I realized just how different the design philosophies between Android and iOS (nee iPhoneOS) are when I wanted Instapaper integration with a good Twitter client. I needed to find an Instapaper app, not a Twitter app with Instapaper integration. I simply didn't grok the universal plugin architecture of Android at first. In hindsight it makes perfect sense—if you're a software developer or tech-geek. It seems insane to require the user to have a mental model that's so intimate with Android's implementation details.

I was going to write about how delighted I was with Android's GMail integration and how much I wish something similar existed on the iPhone. I love GMail. I couldn't care less about other mail configurations (POP or IMAP). Maybe the built-in regular mail app is as clunky as Apple's, but I didn't use it. Kudos to Google for such a seamless integration with not only Gmail, but the calendar and contacts too.

I was going to write about all these little bits of evidence that showed just how different the philosophies are between these two companies and these two platforms. I was going to write about how it's a bit odd to put them in competition with each other because, in a sense, they are really trying to build two different things: Google wants to build the most bad-ass feature-rich, piece of mobile technology there ever was. Apple wants to build a mobile experience.

I was going to write about all of these things, but after attending the keynote at WWDC last week, I realized that there was only one thing that I needed to point to that most effectively highlights the differences between the two:

At the end of this video, you had a room full of five-thousand geeks drying their eyes. You can call the video cheesy or manipulative. You can call it a piece of marketing fluff. But that video was not about protocols, compatibility, specifications or any of the myriad technical details we debate on a daily basis. That video was about telling human stories that we can all relate to.

I haven't yet been to a Google I/O event. I'd really like to attend one. But right, wrong or indifferent, you would never see something like this from Google. I can't think of anything that does a better job of summing up the completely different views that Google and Apple have of the world.

Given the current state of affairs, it's not hard to see why there's a bias against unit-testing in the Cocoa community. ¹ Unit-testing tools, support and idioms within Cocoa are nowhere close to where they are in, say, the Java, .Net, Ruby or Python communities. Compared to those environments, unit-testing in Cocoa is just flat out difficult. I can easily understand why a developer would conclude that the cost and effort of TDD outweighs the benefits.

The Dismal Science

Think of every "best practice" you've ever been exposed to in software. They all come down to cost-benefit tradeoffs in the end. Up-front requirements gathering? It's an attempt to manage change and indemnify certain parties when things go awry. Iterative development? It's merely a short-term review process to evaluate the cost of future development against opportunities in the future. Automated testing? Computers (generally) work more cheaply than humans, so invest in automating repeated tasks so that, over the long-haul, more work is accomplished by cheaper workers. Hell, even the oft-cited goals of code-reuse in object-oriented programming are economic ones. The list goes on and on and on.

But, even as a long-time practitioner of test-driven development, I don't view it as axiomatic. In this big, bad world of ours there have to be cases where the economics simply don't add up, and it simply isn't cost effective to build stuff using TDD.² Returning to Cocoa, I can see how developers come to the conclusion that TDD in Cocoa simply isn't worth the price of admission.

So what makes TDD in Cocoa so expensive? Would things look different if we could change the balance sheet?

Bear-Skins & Stone Knives

Back in the day, OCUnit was the xUnit toolkit of choice for Cocoa programming. In 2006 Apple put OCUnit right into Xcode and TDD received its first official blessing. Now I don't have a beef in particular with OCUnit—it's essentially a faithful implementation of xUnit patterns for Objective-C. What is surprising is how weakly it's integrated with Xcode.

To unit-test, you have to create an entirely new target to execute your unit-tests within. However, you don't run it like a normal executable. It's baked into the build process for that target so that testing errors show up just like build errors. I'll admit that's kind of cute and at least makes an attempt to integrate TDD into the development process. But keeping unit-tests as a separate target means more drop-down flipping in Xcode to go between unit-testing and running my application. Oh and the bloody drop-downs in Xcode…don't even get me started…

The other issue is debugging. If your unit-tests are failing, god help you if you want to find out why. It takes a lot of environment-variable hijinks to be able to actually debug your unit-tests —something that is significantly easier on every other platform I've ever worked on. All of this leaves me with the distinctly uneasy feeling that Apple and the Cocoa community at-large are merely paying lip-service to TDD.

Compare this to how JUnit is integrated into your standard Java IDE or awesome Ruby testing tools like autotest. These tools are so much more immediate and easier to reach for. Xcode looks like it came from the era of the horse and buggy. These kind of tools and this kind of support needs to be a part of the development environment in a more natural way. Right now it's just a primitive, bolted-on afterthought. It's a wonder anyone has the patience to use it.

A New Mentality

Another challenge in Cocoa is figuring out where TDD fits in such a framework-driven environment. To Apple's credit, the frameworks that Cocoa provides do a pretty good job of "making the simple easy and the difficult possible". However because Cocoa is so prescriptive, it can be difficult for developers to stand back and figure out what to test. It's just so easy to just let the idioms fly off the fingers, that testing them seems like a silly exercise.

I think a common conclusion for the the would-be TDD'er is that they often find their tests essentially repeating the implementation. These are most expensive tests to write and maintain. Not only do you end up duplicating the code (thus adding coupling and brittleness), but they also take up a lot of time to write and can be pretty error-prone.

There has been a lot of thought about similar problems in other communities, so why not steal these ideas and apply them to Cocoa development? Surely strategies like inversion of control and mocks would help make TDD in Cocoa an economic possibility. ³

The Cocoa community simply hasn't evolved a good set of testing practices the way others have. Given the benefits I've seen in other environments, it's hard for me to believe that Cocoa and Objective-C are exceptional in this regard. I think that there are ways to do it, we just haven't discovered them yet. I recall from my Java and Ruby days that testing idioms and practices evolved a lot before we got somewhere reasonable. Simply put, the Cocoa world's collective testing skills and knowledge lag severely behind a lot of other languages.

Facing this requires the Cocoa community to face its own isolationist and exceptionalist attitudes. There's nothing so special about Cocoa and Objective-C that conceptually invalidates the effectiveness of TDD. ⁴ Cocoa folks need to look outside of their walled garden to see what others have done. This is not something I've seen much, if any of, in the Cocoa community. Frankly, the dominant attitude seems to be one of snobbery and elitism. It's an unfortunate attitude that holds us all back.

Footnotes

1. Normally it wouldn't be fair to link to a five-year old post and call it "representative" of a community's attitude, but I think for an insular group like Cocoa-nerds, this is totally reasonable.

2. I'm not saying that I, personally, have encountered such a thing, but I don't think it's unreasonable to assume that the possibility exists.
3. There is, of course, a pathological extreme to this line of thinking. That extreme is called Spring.
4. OK, I'll admit that iPhone development is a little different because of having to deal with the device vs. the simulator. Getting tests running on the device is a non-trivial exercise. But I'm not convinced that unit-tests have to run on the device. Yes, there are differences between the real and simulated environments, but those differences should be accounted for in integration tests—a topic I'm not going to address here.

But what if you're integrating with a web service? What's the best way to convey the user's current settings to an external party? I've run into this exact situation a couple of times and wanted to share the approach I like to take.

The Server-Side

The first thing you need to determine is how you'll express the user's current locale settings. If you own the servers-side, you can implement this in whatever way you see fit. Personally, I like to take advantage of as much of the HTTP specification as I can. Rather than overloading every request with a request parameter, I like to use the Accept-Language header. The specification says that the format of the Accept-Language header allows multiple locales with "quality values" optionally given to each one to express order of preference. For example, a valid Accept-Language header might look like this:

Accept-Language: da, en-gb;q=0.8, en;q=0.7
    

In this example, Danish (with no specific region) is preferred first, followed by English (in the Great Britain region), followed by English in any region. The language and region are delicately intertwined in a bunch of different ways. Depending on the combination, the region may affect how words are spelled (for example "armor" in American English vs. "armour" in British English). Not all locales and regions work this way so there are plenty of corner-cases.

Ruby on Rails is my server-side stack of choice these days. Like Cocoa, Rails is pretty good at making it easy to localize and internationalize your application. However, despite Rails claim to being "opinionated", it's surprisingly indecisive about the best way to express language and region preferences. The Rails i18n Guide suggests that you can get the locale as a query parameter (blech), via the domain name (if you've set your application and client up this way), via URL parameters (i.e. as a path segment within a larger URI structure), via built-in user settings, via GeoIP or using the Accept-Language header. As I stated before, I like the latter solution the most. However parsing the header, dealing with the quality scores and then figuring out the best one is something that has already been solved. For that, I like the http_accept_language gem.

In the ApplicationController, I setup a before_filter that uses the gem to parse the Accept-Language header. Since I may not have localized the web server to match a given request, I use the compatible_language_from method to figure out what, if any, language match there is.

before_filter :check_language
    
    def check_language
      params[:locale] = if params[:locale].nil?
        request.compatible_language_from(MyApp::AVAILABLE_LANGUAGES)
      else
        params[:locale]
      end
      I18n.locale = params[:locale]
    end
    

MyApp::AVAILABLE_LANGUAGES is constant array defined in a Rails initializer (in config/intializers) that figures out all of the possible locales I have configured:

module MyApp
      AVAILABLE_LANGUAGES = Dir[RAILS_ROOT + "/config/locales/*"].map do |f|
        File.basename(f).split('.').first
      end
    end
    

Despite my aversion to using request parameters, it can be handy for ad-hoc testing to be able to tack a query parameter on to change the locale. Also, once you've determined the locale, you need to notify the I18n framework as I've done in the last line of the check_language method.

The Client-Side

OK, great. Now we've figure out how to vary the locale on the web-side. How do we extract this information from the device and ship it off in our HTTP headers?

Here you have two tasks: the first is to determine what the value of the Accept-Language header should be and second, to actually make sure you send those values. Since there are several ways of making HTTP requests in iPhoneOS I'm not going to go into the details. Whether you're using NSURLRequest, ASIHTTPRequest, HTTPRiot or your own homegrown thing, presumably you have some reasonable way of setting the Accept-Language header in a consistent way.

To figure out what locale your user is in, you need to consult the NSLocale class. This provides several class methods that yield a NSLocale instance that reflects the user's current settings. If you look at the docs, there are three class-methods that return a NSLocale instance: +systemLocale, +currentLocale and +autoupdatingCurrentLocale. The differences between the three are not apparently obvious and, I think, reflect NSLocale's original desktop heritage.

The +systemLocale method will return a default NSLocale instance when one cannot otherwise be determined. In my experience, I've never seen this return anything but a blank and useless NSLocale instance. Don't bother using it.

The +currentLocale method returns a NSLocale instance that reflects the user's current setting. However there is a bit of language in the docs that I initially found confusing:

Settings you get from this locale do not change as System Preferences are changed so that your operations are consistent. Typically you perform some operations on the returned object and then allow it to be disposed of. Moreover, since the returned object may be cached, you do not need to hold on to it indefinitely.

Huh? To understand what's going on here, we need to take a look at the docs for the +autoupdatingCurrentLocale method:

Settings you get from this locale do change as the user’s settings change (contrast with currentLocale).

Remember, a lot of CocoaTouch came from the desktop Cocoa environment. On the desktop you can open the System Preferences and change your region and language settings anytime while applications are running. However, until iPhone OS 4.0 is released, this is not something you can do on any iPhoneOS device. So the net effect, and this matches my own observations, is that on iPhone OS 3.x +currentLocale and +autoupdatingCurrentLocale are essentially the same. But, if you want to be prepared for the future, go ahead and use the +autoupdatingCurrentLocale method. Once you have a NSLocale instance, you call the -localeIdentifier method on it to get a NSString like en_US or fr_FR.

So if you're like me, you want to test that this actually works. So, naturally, you would figure out how to modify the settings on your phone to yield different responses. What you'll soon discover is that the current locale settings are derived from two separate and independent settings and that various combinations of the two can produce unexpected results.

Start by launching the "Settings" application. Select General ⇢ International. You'll get a screen where you can pick your language, your keyboards and your region format. What may surprise you is that changing the language does not affect the current locale. I'm in the United States so my region format is set to "United States". If I change my language to French, but leave the region format setting alone, all localization within the system and applications will use the French language, but my locale remains en_US.

OK, so NSLocale doesn't quite work as expected, but where does the system store the current language? It turns out the NSLocale provides yet another class method, named +preferredLanguages, that returns an array of language codes as strings. With the settings described above, the first entry in that array is "fr".

If you change your settings where you leave the language set to English, but change the region format to French (in the "France" region), the current locale will now be set to fr_FR and the first entry in the preferredLanguages array will be "en" for English.

What It All Means

Out of the box, Rails doesn't provide full-blown region-specific localization. You can either hack something or use one of many i18n plugins or Globalize2. So if you try to send a language-region combination to Rails (such as en_US), chances are good that you won't have the right localization setup and you'll fall-back to your default locale.

You have two choices: you can make the client send something that the server understands, or teach the server to respect more fine-grained localization values. If you want to go the first route, I would suggest simply querying the +preferredLanguages array and using the first entry as the value for your Accept-Language header. However if your server-side localization can handle it, use the +currentLocale instead so that you can handle regional differences and spelling much better.

The iPad has generated a lot of criticism, scrutiny and analysis in recent months. The debate about the ills and assets of the platform have, and will continue to be, debated endlessly. I think one thing is pretty clear though— whether you like Apple or not, whether you have an iPad or not, whether you think it's the most awesome-est thing ever or not, it is revolutionary device. I don't mean in terms of technical execution (yes the A6 chip is a marvel, the power-consumption is amazing, blah blah blah). What I mean is that somebody with Apple's clout finally stood up and threw away the last three decades of user-interface paradigm and they're laughing all the way to the bank doing it.

So, if you jettison The Way We've Always Done Things ™, what do you replace it with? More importantly, which things from The Old Way are no longer applicable to The New Way? This is a big topic that will take some time to settle itself. We've had this CPU/mouse/keyboard setup for nearly thirty years and we're still learning how to make it work for humans. I don't expect us to settle the micro-debates of touch-interface design for quite a while, but I do want drill down to something pretty specific.

Take a moment and ponder what the mouse really is in a graphic user interface. It's a proxy. It's a loosely-connected device where movement and gestures in one space are translated to movement and gestures in another virtual space. To operate a mouse well, you have to perform constant mental transform operations from the physical world to the world you are trying to manipulate on the screen. The mouse is, effectively, a rather thick layer between you and what you're trying to do.

Exploiting the Gap

As we got better and better with the mouse ¹ software started to exploit this gap between you and your machine. The first place this showed up was in context menus. Once we already had to make a mental stop on the train ride from our brain to the machine, why not make the station more useful to folks? Hell, let's put in some vending machines, and perhaps some couches. We could even install Wi-Fi and people could get even more productive. Pile it on, pile it on.

Taken to an extreme, the mouse becomes the primary means by which we interact with our machines. If you don't believe me, take a look at any professional CAD workstation or that god-awful OpenOffice mouse.

However these are extremes. There isn't necessarily anything wrong with things like extra mice buttons or context menus. But, in a world of touch, where the mental and physical distance between our intentions and execution is much smaller, that thick layer/opportunity is gone. Context menus? How the hell do I "right-touch" the screen? Mouse-overs? Current touch interfaces don't have any proximity sensors ²—either you touch something or you don't. Yet these two very simple interaction models have become a crucial part of the UI vocabulary we have all acquired over the years.

Think about a mouse-over. What is it there for? Mouse-overs offer two things: a preview mechanism allowing you to learn more about something without having to commit to it, and as a backup when a pictorial icon's meaning isn't clear enough. We don't have these on the iPad ³. So as an application designer and builder, how do we give people some notion of what this thing can do? What kind of affordance can we offer?

Spelunking

As you start using and learning an application, you generally have two questions:

• What can this thing do?
• How can I get this to do ______?

These are asked from two opposing angles, but both are about discovering the capabilities of the software. When a user doesn't know what your application can do, how can you build it in a way that they can discover it? An equally important question is how can they explore your application without having to commit to any action, particularly a destructive one?

Let's look at one of my favorite iPad apps in particular, Net News Wire. A nice feature of the application is the ability to progress through all of your unread items with a single tap. When I first got the app, I knew that there had to be a way to do this, but I couldn't find it. There was a mysterious-looking button in the corner of the screen, but I wasn't sure what it was for. I mean, it might do anything. How was I to know what would happen? It wasn't until I saw a presentation by Brent Simmons and Brad Ellis on the design of NNW that I actually found out where that button was. You can't really figure out what it does unless you poke it. If you poke that button, there isn't a corresponding "undo" button for that action.

That's not a dig against Brent and Brad, I think they did a wonderful job with NNW. But it does highlight how difficult it can be to convey such a feature to the user. I think it may be one of the biggest challenges on the platform. How do we let the user know what our application can do?

Let's be honest here, if you're relying on documentation to teach the user the basics, you've already lost the battle. Documentation is fine for really deep, detailed information (see OmniGraffle on the iPad for an example). But requiring a user to read the Owner's Manual before they can even use your software went out about the same time as floppy disks. On the iPad an up-front documentation requirement is just laughable.

Little Gets Big

One final thought is that although the iPhone and iPad obviously share the same DNA, the difference in size makes this acutely problematic on the iPad. On the iPhone you simply can't cram that much stuff into an application. Read the iPhone Human Interface Guidelines and you'll quickly get the message that on the iPhone, less is more. By and large, the best iPhone apps simply don't have that many features integrated into them. This gives application designers more freedom to make clear the intent of the features that are implemented.

On the iPad, it's a different story. There's so much more real-estate. Running an iPhone application on the iPad is so laughably awkward that it starkly highlights just how different the platforms really are. So now, as iPad application builders, we have more space than ever. While we're freed from the space constraints of the iPhone, we now have a challenge (and responsibility) to use it effectively. It's not hard to imagine some developers embracing these new green fields to produce some truly awful interfaces. Please, don't be one of them.

Tangents

1. Make no mistake here, we've had to train ourselves to use the mouse. Just watch your grandparents struggle with a mouse and you'll realize how un-intuitive the device really is.

2. I'm not even sure if they did that it would be such a good idea. Requiring that level of fine-finger dexterity immediately makes such a device exclusive to the young and facile.

3. OK, there is a common trick of touching and holding a hyperlink long enough to get a menu of options.

Size and Geometry

There is a axiom in military science that says no plan survives contact with the enemy. In a similar vein, no slide layout survives contact with an actual display screen. As we craft presentations it's easy to get so locked into looking at our screens that we often forget about the context of the room in which it will be given. There are two important things to consider about the room: how far back the audience stretches from the screen and the angle of the audience. In a room with a "deep" audience and shallow angle, anywhere from the bottom 20% - 30% of the screen will only be visible to the front row. Some professional presenters make it a matter of habit to simply avoid the bottom quarter of the screen. Think of this as a creative constraint to embrace and help you focus your message.

The dimensions of the audience also affect the size of the content in your slides —specifically the font size you choose. Somehow font sizes that look like billboards on your laptop are diminished to footnotes once they are projected. In practice, it's pretty hard to make text too big. If you have a problem fitting what you want to say on the slide, perhaps you need to reconsider the phrasing.

Aside from direct quotations, I find that it's rarely useful to have full paragraphs of text in your slides. You want your audience to be focused on you, not your slides. Think of the slides as simply the supporting cast to what you're telling the audience verbally. How can you craft your slides to complement what you're trying to convey?

Live Demos

Slides can only take us so far, especially for technical presentations. Sometimes we need to turn away from the "what", and explain the "how" of something. We want to demonstrate how something works. For coders, this often means a live coding or tool demonstration. If you're trying to convince your audience that technology XYZ is the greatest thing since sliced bread and will save them all sorts of time and headaches, you want to be able to show it, not just tell it.

Fantastic. Good for you. Here's the problem: live coding demos are a recipe for disaster. I can't think of a better scenario that demonstrates Murphy's law than trying to execute a live coding demo. You may not have an internet connection (oops, there goes the ubiquitous twitter client demo). You may have installed some beta software on the plane ride out and, unknowingly, destroyed your development environment. The list of things that could go wrong is unbounded.

The other problem with live demos is the switching in and out of the presentation software you're using. Not only is it visually jarring for the audience, but there is invariably some down-time as you switch displays and try to figure out which windows are where. Don't give your audience the opportunity to tune you out during these noisy transitions.

So, just like you wouldn't create your slides on-the-fly as you present, you shouldn't demonstrate live without a net. Instead, record a video of it and embed it in your presentation. With a video you get a chance to edit out all of the hiccups and noise of the demonstration process. Mistyped something? Got a compilation error? Tools are running really slow for some reason? Great, just edit those frames out.

Leaving these blemishes in only gives the third-grader part of your audience's brain time to sneak in and distract them. You want to keep your audience engaged with you every step of the way. Introducing these interruptions into the continuity of your presentation breaks the audience's concentration and your flow.

Once you get your video captured and edited, practice speaking over it. Really make sure that what you say matches well with the video. Get the timing down and stay focused on what you're trying to show. Is there something you want to say that you isn't in the video? Get back in there and record the right video. Don't half-ass it. When you put the time in to make a good supporting video, you keep your audience longer.

Keynote Specifics

If you're using Apple's Keynote, here are a couple of tips. Once you've captured and edited your video, simply drag the video object onto a new slide. Open the inspector palette and select the last tab with the Quicktime logo on it. Select the checkbox labeled "Start movie on click". This way your video won't start running before you're ready to talk about it.

Also, with a stock Apple remote, you can pause and resume your video as needed. This is especially helpful when you have a multi-step process to explain that you want to cover in bite-sized chunks. This means that you should consider pause-points in your capture and editing process. Practicing your speech along with the video will help you figure out where the natural breakpoints are. Alternatively, you could split the process up into multiple videos on different slides.

Eschew Style

Customization and personal style are well and good. On your machine you should feel free to tweak every little setting to your heart's content. But, please, don't force these styles on your audience. Ditch the alpha blend on your terminal. Turn off your crazy four-line shell prompt. Pick a color theme in your editor that is easy to read in a large font. Pick a display font that is appropriate for large projection, not just what you prefer.

Make those fonts BIG. Just like your regular speaking voice needs to be amplified to carry to the back of the room, so does the visual volume of your content. When recording your coding demos, pick unusually large font sizes and take up all of the screen. Also, when capturing your video, try to keep the capture frame focused on the essential parts of the demo. For example, if you're only editing text, don't clutter the video capture up with a toolbar full of things you won't use. Remember, keep it focused! Don't give your audience a chance to get distracted by clutter.

Keep It Focused

When presenting a technical topic, it can be frustrating having to jam a large amount of content into a small space. There's just so much to share! How can I possibly discuss Core Animation in forty-five minutes? The answer is, you don't. What you can do is give your audience a taste of the topic you're trying to present. Give them enough information for them to decide if they want to pursue more on their own. Give them enough terminology and background concepts to help them continue the journey.

Presentations are not a good vehicle for in-depth training. Books or workshops are much more appropriate for real hands-on learning. Presentations are good at giving people a mental roadmap to get started. Get them excited by your topic, but don't exhaust them with it.

Just look at that. Do you think that fits on anything but a large display? I can't even see most of these icons on my built-in MacBook display. Only humble little Finder, with its spartan set of menus, yields enough real estate to display all of these items.

Sadly, I had gotten into the habit of knowing which applications hid status bar items from me and would automatically switch to Finder just to reveal them. It was the classic mistake we all make of giving in to bad design instead of addressing the problem.

So I waged a small holy-war this morning to kick out all but the essential applications from my status bar. I feel a little better that I've reclaimed my territory, but can't help but feel frustrated that I'll have to do it again soon. Why? Because, somehow, application developers got it in their heads that they all have to have menu bar items.

I want this to stop. Right now. Please. Do it for the children.

I can't help but think that this reflects a dangerous attitude underlying a lot of applications. Stop for a moment and think of the reasons why an application provides a menu bar item. Menu bar items exist for applications running in the background. These come in two flavors: applications with no other user interface (like a window or a preference pane), and ones that do.I don't have much of a beef for the ones that can only provide a menu bar item as their sole interaction mechanism. It's the applications that do have a main UI that I want to give a stern talking-to.

In a windowing system users get to choose what they focus on (save for a few cases with alerts and modal dialogs and such). Applications are supposed to do the bidding of the user. When I switch away from application XYZ, it means I have something more important I need to do. Having that application pop back in my face drives me nuts. It drives everyone nuts. I think most folks would agree that this is a Bad Experience.

But some applications want to keep your attention. So they compromise by creating a little persistent bit of goo and sticking it in the menu bar. Because these apps are so fabulous, how could you not want them around all the time?

The best menu bar items are ones that provide the essential capabilities of the application and give you a quick way to expand to the full application. I run iStat menus because it gives me a lot of information in the small space it takes up. Why is my machine so slow? Ah, the CPUs are pegged. Why are they pegged? Well, just click the menu item and see that application XYZ is taking up 90% of my cycles. It's a great example progressive disclosure.

Some applications like iShowU or Little Snapper provide a menu bar item because they need to get out of your way. You don't want to screen capture the screen capture tool itself, right? For these kinds of apps that provide functionality that spans multiple applications, I think this is okay.

What I can't stand are applications that provide a menu bar item, then keep it there after I've quit the application. Evernote is a good example of this. You have to really go out of your way to make Evernote not do this. Worse yet, its default behavior is to install itself as a startup item so that it's sitting there in your menu bar until you take time out of your busy day to turn it off. Boo. Hiss.

I like Evernote quite a bit, but I get the impression its creators think it's so special that it deserves to be in your face, all the time. It's like that annoying guy you got stuck talking to at the company party that you can't rid yourself of. Go away.

The worst case of this is VMWare. I can't, for the life of me, figure out how to disable VMWare's menu bar item. For folks that need to run a lot of non-Mac apps I can see the advantage. The Unity stuff in VMWare is pretty freaking cool. But if you're not one of those people, it's pretty rude to have this whole other application-launcher sitting there, taking up space, that you only occasionally use. Shame on you VMWare. I'm happy to pay for a license because I think it's a great product, but that doesn't mean I want to see your goddamned menu item in my status bar all the time.

So if you're building a Mac application and considering a menu bar item, please consider the following advice. If you're building a background utility with no other UI, think hard about how much the user needs to interact with your application. Is it something they need to fiddle with more than once a session? If not, try to make a Preference Pane and leave it at that. If it's something the user is going to use a lot, or displays some kind of realtime information (à la iStat menus) then adding to the menu bar is okay.

If you're building an application with a main UI component, think long and hard about the necessity of menu bar item. If it can integrate with other applications (like Little Snapper) there's a reasonable case to be made for providing a menu bar item. Even then, you should consider providing a Services menu item and/or a global keyboard shortcut. This is an especially good idea if your application only provides one or two actions that users can take. It's hard to justify a two-item menu bar item, so sometimes you see applications put other non-essential items in there like Preferences or an About dialog. Don't do this.

For these sorts of applications, the menu bar item should always be optional. The app should still be able to do what it needs to without having that menu bar item enabled. Also, leave it off by default and make it easy for the user to turn it on. Make it an opt-in feature rather than an opt-out feature. As much as I love Tweetie, the opt-out nature of its menu bar item is the kind of stuff that drives me nuts. The option is enabled (or not disabled?) and the language is confusing. Worse yet, it doesn't take effect until you restart the application.

Finally, unless you have some kind of special papal dispensation, never, ever, ever keep your menu bar item running when the application is gone. OK, technically if there's a menu bar item the application is still running, but that's a distinction that user's don't care about (nor should they have to).

There's been a long, ongoing debate about the necessity of optimizing the performance of applications. As horsepower has increased, programmers have been afforded some laziness in optimizing their applications. I only have a problem with that in extreme cases because hardware power has increased at such an impressive rate. However, our displays aren't growing at the same rate. We need to treat that screen real estate as precious. The menu bar is a great place to put global, cross-application functionality, but there's only so much space for all of these applications. Think long and hard about the necessity of taking up that space and always put the power in the users' hands to choose.

Since her death, I've been tossed into the deep-end of estate-management as I try to clean up the remnants of her financial and legal life. It's amazing how much a person leaves behind. There are all sorts of things to clean up like calling her dentist and canceling a pending appointment, or telling the local pharmacy that she won't ever be coming in again to pick up her prescription.

In the last month, I've realized I'm not the same person I used to be. It sounds trite, but this event has given me a serious perspective adjustment. Things that used to seem scary to me now seem insignificant. Things I always wanted to do but put off for some day in the future, get addressed now. Life is short. You never know what can happen.

By nature, my wife and I inveterate planners. My god, I started an IRA when I was 21. Who the hell does that? But now it's time to think more about the present. We haven't completely abandoned our careful planning, but we've stopped putting off things we want to do.

One of those things has been a growing desire to be my own boss. For the longest time I could never imagine running my own show. There was too much work, stress and responsibility. I was happy trading independence for the comfort and security of a steady paycheck. But over the last few years I've been feeling a deep and acute dissatisfaction working for someone else. I came to the conclusion that I wasn't going to be happy until I stopped working on other people's ideas and took a shot (or two) at working on my own.

So now it's time to stop being so unhappy about it. As of May, I'm on my own. My plan is to build a small product-development company. I don't need to make scads of money, I just want to support my family. I want to work on things that I deeply care about. I want to spend my time doing things that matter to me, and as little time as possible on "shoulds", "musts", and "have-tos". I'll pick up contracting work as needed (tell all your friends), but the long-term plan is to live off of product revenue. I guess some people might call it a "lifestyle business". I call it dropping out of the rat-race and living the life more fully.

How am I going to do this? What's my plan? Beats the hell out of me. I'll figure it out as I go. All I know is that everything I've gone through in the last month has made me realize that it's going to work out fine. There's nothing that could happen that would equal what I've recently gone through.

So what about the title of this post? What exactly am I so thankful for? Well, certainly not about the loss of my mother. Nothing could replace her. Nothing will bring her back. She was my moral compass. She was the greatest teacher I ever had. I didn't really appreciate the tools she left me with until she was gone. Now I realize that she left me with everything I needed to cope with this tragedy as well as being able to move beyond it. I don't know if I would ever have been shaken out of my day-to-day routine without her loss. I wouldn't trade this epiphany for her, but I don't have that option. What I can do is move forward and live a life that honors her memory.

My mom was a survivor. She survived the breakup of my family. She survived cancer. She survived my brother's death. After her divorce she rebuilt her life from scratch, eventually becoming an assistant Dean at the School of Education at the University of Oregon. Her crowning achievement was completion of school's new building on the U of O campus. Over the span of nearly a decade she raised funds from private donors—mostly teachers. But the final dollar amount isn't what's impressive about her work. What's impressive is how she took a group of people, turned them into a community and got them to believe in themselves. Mom, I wish I could have told you just how proud I am of you.

So, mom, this life is for you. You always had a strong independent streak. You didn't get a chance to enjoy the future that you worked so hard for. You gave me so much. The best way I can think to pay it back, is to live the life you would have wanted for me, and to pass that on to my daughter.

Thanks, mom. Thank you, so very much.

First, is the script that selects the SDK for the device:

tell application "Xcode"
      set myProject to active project document
      set active SDK of myProject to "iphoneos3.1.3"
    end tell
    

Next up, the script that selects the simulator SDK:

tell application "Xcode"
      set myProject to active project document
      set active SDK of myProject to "iphonesimulator3.1.3"
    end tell
    

Now, open up Xcode and and select "Edit User Scripts..." in the scripts menu:

This opens the script organizer. In the left-hand pane is a list of all the script groups. Which group you put a script in effects what submenu they are available from in the main scripts menu. But we like shortcut keys so who cares where it goes? I put these in the "Build" group, but feel free to put them wherever makes the most sense to you. Select the group you want to add these scripts to and click the "+" button and choose "Add Script File…":

Use the standard Finder dialog to find the scripts you just downloaded and select one of them. To set the shortcut key for the script, double-click the right-hand column of the newly-added script. For the simulator script I use ⌃⌘⎇S and ⌃⌘⎇D for the device script. You can try out different combinations. If you come up with a key-chord that clobbers an existing key-binding Xcode will give you a warning at the bottom of the window.

One thing to be aware of is that while you're setting the shortcut key for the script all of your keyboard input is directed to that field. So you can't use ESC to cancel or Enter to set the value. You need to grab that infernal mouse and click somewhere else.

Before you close the dialog, be sure to set the "Output" drop-down menu to "Discard Output" otherwise Xcode will paste some odd gibberish into your current file. And, just in case there's some kind of unforseeable error, set "Errors" drop down to "Display in Alert". Once you have both scripts configured you can close the dialog and try your new hot-keys out.

I don't claim to have any proficiency with AppleScript, so there may very well be a better way to do this. Certainly one thing that is brittle about these scripts is how they are hard-coded to a particular version of the SDK. Whenever a new SDK is released I'll have to update the strings in the script. Oh well, it beats having to reach for the mouse several more times a day.

So I've setup a tiny bit of zsh-fu to automatically switch which ruby implementation to use based on the directory that I'm in. To do this, one needs to take advantage of zsh's hooks, particular the one invoked when you change directories. By default, you can simply declare a function named chdir() and it will automatically be invoked whenever you change directories. Mine looks like this:

chpwd() {
        [[ -t 1 ]] || return
        case $TERM in
          sun-cmd) print -Pn "\e]l%~\e\\"
            ;;
          *xterm*|rxvt|(dt|k|E)term) print -Pn "\e]2;%3~\a"
            ;;
        esac
    }
    

This function just prints out the name of the directory I've change into and it's found in a lot of zsh distributions. To keep things clean I want to have a separate function that just manages the RVM updating. For the moment, let's ignore the fact that I already have an existing chdir() function, and look at the hook function I created to invoke RVM:

chpwd_check_rvm() {
        current_version=$(rvm info | grep " version:" | cut -d '"' -f2)
        dir=$(pwd)
        while [ "${dir}" != "" ]; do
            cfg="${dir}/.rvminfo"
            if [ -f ${cfg} ]; then
                want_version=$(cat ${cfg})
                if [ "${want_version}" != "${current_version}" ]; then
                    rvm use ${want_version}
                fi
                break
            else
                dir=${dir%/*}
            fi
        done
    }
    

This function looks for a file named .rvminfo in the current directory (this function is invoked after we've changed directories). If the file isn't found in the new directory, it searches upward through each parent directory to see if it can find one. If it's found, the function invokes rvm use with the version string found in that file.

$ cat .rvminfo
    1.8.6
    

There's also some additional checking to avoid extra rvm invocations if we're already set to the desired version. This is done more to reduce the chatter in the shell than for performance reasons.

Now let's get back to invoking multiple hook functions. If you have more than one function you'd like to have called when you change directories, you can declare these in an array named chpwd_functions. After declaring the chpwd_check_rvm() function above, having two functions for the cd hook is trivial. I just add this declaration to my ~/.zshrc file:

#--- chpwd_functions
    chpwd_functions=( chpwd_check_rvm chpwd )
    

Now, whenever I change into a directory with a .rvminfo file, my hooks are automatically executed and I get the right ruby version:

[island] ~/Development: cd alexvollmer.com 
    <i> Now using ruby 1.8.6 p383 </i>
    

One final note is that I put all of my specific functions in the ~/.zsh/functions directory, and automatically load all of them in my ~/.zshrc file with the snippet below. I do all of this before I declare the chpwd_functions array.

#--- Shell Functions ---
    #
    fpath+=(
      ${HOME}/.zsh/functions
    )
    
    autoload -U ~/.zsh/functions/*(:t)
    

You may want to organize your functions differently. Just be aware that zsh needs to know about your functions before you can put them in the chpwd_functions array.

UPDATE (3/1/2010): Per comments below from RVM's creator, Wayne Seguin, RVM actually comes with it's own built-in way of handling this. Somehow I completely missed the rvmrc documentation for this feature (how embarassing!). I'm usually inclined to avoid re-inventing the wheel, but there are two small things I like about the way I implemented this. First, the solution proposed here doesn't redefine the cd command, which feels a bit like duck-punching the shell to me. I don't know if bash has the same kind of hooks that zsh does, so the implementation is understandable. As for me, I like the hooks better. Second, the solution I show here works for any sub-directory within a project that has a specific RVM setting. That said, I'll probably just use what's already been implemented in RVM.

I chose to use Nanoc as the site-generator. Originally I was going to go with Jekyll. I really liked its simplicity and the fact that it had prescribed locations for everything. However, I still wanted to maintain some backward-compatability with the old site. This meant a little bending of Jekyll's rules and, frankly, Jekyll just isn't as configurable and flexible as Nanoc. That's not a dig against Jekyll, nor is it an endorsement of Nanoc. Nanoc is a general-purpose solution for generating static sites, but it's not necessarily optimized for blogging. But with Nanoc I can use HAML, SASS and other Rails-isms with which I'm already familiar. If you want to see the gory details, I've put the whole thing up on GitHub.

Sadly, this ended up taking much longer than I anticipated. First, I had to import the old posts from Wordpress. The database schema for Wordpress is pretty wacky looking when you're used to working with Rails. Thankfully, Jekyll already gone down this path and I used their Wordpress importer as inspiration for writing my own. Along the way I discovered what great gem Sequel is. A large part of the importing process was converting from all of the jacked-up formats I've amassed over the years to Markdown. It took a lot of iterations to catch every weird character-encoding and formatting issue.

Another large effort was creating the RSS feeds. Now Nanoc has some support for Atom, but it didn't quite do what I wanted. I was trying to match what Wordpress was generating for me so I basically re-implemented feed-generation from scratch.

Moving old comments into Disqus took about a week to get right. As a user, the Disqus service is pretty cool. As a web API, it's a little…ahem… lacking. If you're used to well-constructed REST APIs, you'd best drink something stiff before settling in with the API docs. The disqus gem borders on useless. So, I ended having to write my own importer with httparty. It took several iterations to really get this nailed and the process was pretty frustrating.

The sad part about this, is that originally I was on the fence about adding comments at all. Initially I thought I'd try it out and see if I liked it. But I'm a geek, right? I couldn't get it working right away and it drove me nuts! I had to figure it out. So eventually I figured out the magic recipe, and now I'm wondering if I even really wanted comments integrated at all. Oh well.

One trick that I found very helpful throughout this process was creating some scripts that dropped me into an IRB session with the Nanoc blog or Disqus API objects already initialized. This was inspired by the Rails console which I find invaluable for tinkering with things in real-time. For example, while I was dickering with the Nanoc internals, I found this console script to be quite helpful:

#!/usr/bin/env ruby
    
    require "irb"
    require "rubygems"
    require "nanoc3"
    load "lib/helpers.rb"
    
    @site = Nanoc3::Site.new(".")
    @site.load_data
    
    IRB.start
    

Now, Wordpress and I differ on what constitutes a sensible URL scheme. So that last bit I needed was to concoct some redirect rules to port the old URL scheme to the new one. Currently, this site is hosted on http://dreamhost.com which, provides support for .htaccess files. With a little Apache-fu I was able to get most of the old WP URLs to redirect correctly:

RewriteEngine ON
    RewriteRule ^index.php/feed/ /feeds/rss_2_0.xml [R]
    RewriteRule ^index.php/feed/rss/ /feeds/rss_0_9_2.xml [R]
    RewriteRule ^index.php/feed/atom/ /feeds/atom.xml [R]
    RewriteRule ^index.php/(.*) /posts/$1 [R]
    
    ErrorDocument 404 /missing.html
    

Well, that took longer than I thought. I can't say that it wasn't educational. Now, what was I supposed be doing?

Those that damned the iPad seemed to come in two flavors: those that saw the iPad as the embodiment of corporate greed and walled gardens, and those that thought the device didn't measure up to their personal expectations.

There is a sizable (and growing) population of folks that simply will not purchase an Apple product or participate in its eco-system for various reasons. I think that's fine. While I personally enjoy their products, I don't believe that everyone has to agree with me. I get no less enjoyment out of my MacBook or iPhone by someone else having a Lenovo ThinkPad or an Android Phone.

Unfortunately, that view isn't shared by many. Take the flame-baiting in the aftermath of the iPad announcement and replace the term "iPad" with "gay marriage" and the arguments look nearly identical. Gay marriage is so upsetting to conservatives that they can't imagine being in a world where it exists. It's not that gays are coming into their homes and performing unspeakable acts, it's that they can't imagine even being on the same planet with them. So it goes with the iPad—there are the FSF types who can't imagine living in a world where Apple controls every single thing that goes on a device.

I'm not aware of anything in the President's address that said every man, woman and child in the U.S. was going to be forced to use an iPad. As far as I know, people are still free to choose whether or not they wish to purchase and iPad and wander about in Apple's walled garden. Don't want an iPad? Fabulous. Don't buy one.

Folks, it's a big world out there. There doesn't have to be a single winner. Even as Apple trumpets having over 50 million touch devices, that pales in comparison to the worldwide population of those who could afford such devices. That's even without considering the fact that if you own one iPod/iPhone, you probably own another. In terms of real numbers, I don't see a growing "collectivization" movement to dress everyone in Apple products. If Apple were interested in market share, they would charge a lot less for their products.

Besides calling the iPad a freedom-killer, the other category of critiques were that the device was simply disappointing. This reaction particularly surprises me. Yes, there are certainly quibbles to make about various features of the device (no camera, no multitasking, etc.) but the dismissive observations that the iPad is simply "a big iPhone" I think misses the point. Yeah, it is a big iPhone, which has been a huge success by any measurement. If you love the iPhone, then the things you most wish it had were more of are screen real-estate and speed. So making one that is bigger and faster is totally logical. What did people expect? Roomba integration? A toaster? 3D goggles?

ReadWriteWeb attempted to break down the anger with this graph:

Looking at this chart, I would categorize 38% of the complaints as just pure snark. The next 27% that dismiss it as a "big iPhone" are missing the point. The remaining complaints are all valid and worth talking about a bit:

No Flash

Hooray, I say. I don't run Flash on my desktop or laptop if I can help it. I simply do not like Flash. It rarely enhances my experience on a site and often pegs my CPUs for no good reason. Yes, I'm aware that Adobe has to do a lot of video codec work without the aid of hardware, but that doesn't address the jarring user experience that is Flash. I have not missed Flash on the iPhone either. If you want Flash, don't buy a iPod, iPhone or iPad. It's as simple as that.

Oh, and if you're running a Mac and feel the same way, do yourself a favor and install the Click to Flash plugin, which lets you ignore Flash on the web completely.

No Multitasking

I'm on the fence about this one. I'm not convinced that multi-tasking is essential for a device like the iPhone or the iPad. The truth is, I have not had many instances where I thought, "Damn! If only I could run two apps at once!" I think a lot of people think they want it because their computer has this capability.

I think that's the point that is being missed. The iPad is not, in any way, being put forth as a general computing device. In fact, to categorize it as a "tablet PC" is to completely misunderstand the iPad's underlying design philosophy. When I watched the presentation and demonstration video what I saw was easy execution of tasks. Nobody was demonstrating bit-rates, frames-per-second, or tail-call optimization. The underlying technology and "platform" are simply the means to an end.

This is a different enough device that people will have to change their thinking about their relationship with their computers. I don't think it means a comprimising relationship, but one that is different from today. Fraser Speirs eloquently called the inability to comprehend this shift as Future Shock.

Lock-In

Another major complaint of the iPad and the entire Apple/iTunes ecosystem is that it is a closed system. Users don't get to mix and match devices freely from other manufacturers. The party-line from Apple is that they want to control every aspect to provide the best user experience. Detractors argue that it is part of Apple's plan to own all of your content like Big Brother (as Microsoft has attempted to in the past).

I get why people feel uncomfortable with handing Apple control of their stuff. Personally, I have a lot of Apple products in my house and I'm happy to give them that control because, for me, it is a better experience. I will happily trade some vague notion of "freedom" for stuff that just works. Endless fiddling and integration of disparate technologies isn't freedom at all—it's an enslavement of the most precious resource I have, which is my time.

Not everyone is happy with that and they prefer an alternative. Hey, no problem. Live and let live. You want to run Boxee on Linux connected to an iRiver. More power to you. I won't sneer at you or call you a fool. If it makes you happy, your choices don't affect me in the slightest.

As an aside, if anyone complaining about the lack of Flash is also complaining about lock-in, that person needs to be hit repeatedly with a rake.

No Camera

Sigh. Yeah, this one is a bit disappointing. I don't really care so much about having for video chatting (which I do maybe once a year). But I really like using a camera to capture things I would otherwise have to take a lot of effort to write down. How cool would it be to have your iPad in a brainstorming session where you can still jot a few notes down then associate a snapshot of the whiteboard?

Now there is an interesting design conundrum here. Do you put the camera on the front or the back? Depending on the tasks you expect to perform, this makes a huge difference. If you want people to see you, you put it on the front and figure out how to give the user the user visual feedback so they can position the camera.

If you want to take pictures of other things, you put the camera on the back and use the video display as the viewfinder. However you can't reasonably reverse the tasks for these two positions. OK, so do you put two cameras on? I don't know. How do you control them? Does each bit of software implicitly decide which camera to enable? Does the user have to decide?

I don't know the answer, but I would be very surprised if a camera didn't show up in the next version of the device. They put cameras in the iPod Nanos for cryin' out loud. Surely they can figure out how to stick one in an iPad.

More AT & T

The amount of bad press AT & T has received via the iPhone has to make them wonder what sort of devil they've made a deal with. Are there serious issues with AT & T? You betcha. Do they effect everyone? I'm not convinced. Coverage in San Francisco and Manhattan seem acutely poor, but in my corner of the Northwest coverage is pretty damn good.

The non-iPhone types love to make fun of AT&T, fueled largely (I think) by schadenfreude. I'd be very curious to see how other carriers handle a similar load. I used to work in that industry and the mentality among carriers is all the same. They all want to wring every penny they can out of you by charging for every bit you use. Let's not kid ourselves.

Kudos to Apple for making a WiFi-only version. For me, this is a pretty compelling configuration. I'm an urban kid. There are few places I go where I can't get some kind of access to WiFi. In the cases where I can't get WiFi, I have an iPhone. Plus, I imagine using an iPad primarily at home, or on vacation, but not for commuting.

I'm no AT & T apologist—there are some serious issues they need address— but I'm not convinced that this makes the iPad a "failure".

Conclusion

Am I an Apple apologist? I hope not, but people are notoriously bad judges of their own character. What I can say is that there are certainly things about the company that frustrate both as a user and developer. However, I am still more delighted by their products than I am frustrated with them. When that balance changes, I'll move on to something else.

I'm pretty jazzed about this device, both as a consumer and as a developer. What sucked me into the iPhone was how it just oozed with utility. Not in terms of potential energy, but in that it fulfills the notion of computers as and extension of our own minds. The iPad just takes the next step forward. I can't wait to see how it all unfolds.

P.S. Just I'm glad to see the iPhone finally get some competition, I'm also looking forward to somebody putting up a good fight against the iPad.

After returning from WWDC I was inspired to try this technique out and after playing with the idea for a bit, it was clear that there was a lot of repetition in the process. To really make prototyping cheap, we need a simple framework that makes the process fast and easy. Enter Briefs, a lightweight iPhone application that allows you to embed and run simple prototypes built with nothing more than images and a property list.

With Briefs, you don't write a separate application for the device (imagine the provisioning headaches), but instead embed separate prototypes (called "briefs") into a single Briefs application. Briefs is still pretty new and I expect to see (and contribute) many enhancements, but if you like this style of application development, it's certainly worth spending some time with Briefs.

Let's look at how it works. We're going to start with the goal of prototyping the built-in Photos application. Now obviously, nobody is going to actually re-write the Photos app, but since we all have familiarity with it, it's a good example to see how a working application can be expressed as a Briefs prototype.

Getting Briefs

Before we do anything, we need to get Briefs on our local machine. Briefs is distributed in source-only form via GitHub. You'll need to find a nice place to check out the two main Briefs project, then do the following:

$ git clone git://github.com/capttaco/Briefs.git
    $ git clone git://github.com/capttaco/Briefs-util.git
    $ cd Briefs
    $ git submodule update -i
    

In order to put our first Brief together, we need to build a command-line tool that is part of the Briefs project. Open the Xcode project in the Briefs folder (Briefs.xcodeproj) and select the "Briefs Compactor" target and set the architecture to "Base SDK".

Now execute the target to build the compact-briefs executable. When it's finished, Xcode will put the final binary in build/Debug/compact-briefs in the Briefs project directory. Copy this file to a location in your $PATH. Personally I like to keep such stuff in ~/bin. We'll need this command in a little bit.

How it Works

Before we dive into our project, we should get a little terminology under our belt. Each application prototype you create is called a "Brief". Each Brief is composed of one or more "Scenes". Scenes usually have an associated image, so you can think of a scene and a screen in the application as being more or less equivalent.

Scenes can also have any number of "Actors", which are regions on the screen that perform some action when touched. For example, to mimic navigating to a new view via a view controller, a region on a table view could be rigged to an action that slides another scene into place.

Briefs are expressed as Property List (Plist) files in the Briefs application. If you like writing XML or are proficient with the Property List Editor application, then you're golden. Otherwise there is another format in which you can write your brief that compiles down to a final Plist. This is the route we'll explore in our prototype.

Protyping Photos.app

Generate Image Files

Now let's generate the image files for our prototype. In the case of the existing Photos app, I just screen-captured the running application on the simulator. Obviously, you won't be in the same position with a new application, so you'll need to generate some images by hand.

If you use OmniGraffle, there are several good iPhone stencil sets on Graffletopia. If you prefer Photoshop, checkout the exhaustive iPhone GUI PSD. Either one of them will generate the appropriate files. There's no requirement that the image files look like the iPhone. You could use a tool like Balsamiq or scan hand-drawn sketches to generate very simple wireframe sketches. Regardless, all you need to do is produce PNG, GIF or JPEG files that fit within the normal iPhone dimensions (320 x 480).

Let's create a folder called "photos app" and put our image files in it. I've created a tar-ball of this entire project, which you can download, explore and run.

Writing the Brief

Let's think a bit about the flow of our application. We're only going to implement a sub-set of Photo.app's full navigation—we'll just implement the flow for the Hawaii photo album. We have a top-level screen that shows all of the photo albums. From there the user can select "Hawaii" and drill into a display of a few images in the album. Touching any of the images will take the user into the image browser where users can go back and forth between the various images. Each screen has a common "back" button in the top-left. The overall screen layout and flow looks like this:

The pink boxes highlight the touchable areas in each scene. The arrows between them show which scene a particular area navigates to. In Briefs-parlance, each screen is a "Scene" and each pink box is an "Actor".

Let's step through the Brief (the photos.bs file), a bit at a time. If you get lost, refer back to the application flow diagram above.

First we declare that our starting scene is the "Home" scene. Next, we define that scene with the image home.png as the background image and one actor that will navigate to the next scene. The position and size of this actor corresponds to the table cell labeled "Hawaii". If we wanted to support the table rows, we'd need to add an actor for each one.

When the user touches the "Hawaii" row, we'll transition the "Hawaii" scene (defined shortly) by executing a "push left" animation. This will mimic the standard animation of pushing a new view controller onto the navigation controller stack:

start:Home
    scene:Home
      image:home.png
      actor:HawaiiPictures 
        position:0,123 
        wh:320,53
        action:goto(Hawaii, push left)
    

Next, we'll define a scene that shows the thumbnail images for each photo in the Hawaii album. This scene will display the hawaii.png background image and has an actor for the return navigation button in the upper-left as well as for each thumbnail image. Note the differences in the direction of the push animation in the actors. The "back" button pushes right (like the normal return navigation animation), while the thumbnails push left.

scene:Hawaii
      image:hawaii.png
      actor:Home 
        position:17,30 
        wh:87,24 
        action:goto(Home, push right)
      actor:image1 
        position:5,69 
        wh:72,72 
        action:goto(Image1, push left)
      actor:image2
        position:84,70
        wh:72,72
        action:goto(Image2, push left)
      actor:image3
        position:162,70
        wh:72,72
        action:goto(Image3, push left)
    

Now we'll define the scenes for each of the three images. Each image scene has an actor for the return navigation button in the top-left. The first image also has two actors to navigate to the next image. The NextImage actor is for the navigation button in the toolbar, whereas the NextImageSwipe actor covers the right side of the image itself. This is done to mimic the swiping motion normally used to navigate photos. Right now, Briefs only supports single-touch interactions so it won't behave exactly like the real application in supporting swipes.

scene:Image1
      image:image1.png
      actor:NavigateBack 
        position:14,33 
        wh:44,23 
        action:goto(Hawaii, push right)
      actor:NextImage 
        position:202,450 
        wh:23,17 
        action:goto(Image2, push left)
      actor:NextImageSwipe 
        position:156,60 
        wh:160,380 
        action:goto(Image2, push left)
    

The "Image2" scene is a lot like the "Image1" scene, except that it has button and swipe actors for both the previous and next images:

scene:Image2
      image:image2.png
      actor:NavigateBack 
        position:14,33 
        wh:44,23 
        action:goto(Hawaii, push right)
      actor:PreviousImage 
        position:94,452 
        wh:23,17 
        action:goto(Image1, push right)
      actor:PreviousImageSwipe 
        position:0,60 
        wh:160,380 
        action:goto(Image1, push right)
      actor:NextImage 
        position:202,450 
        wh:23,17 
        action:goto(Image3, push left)
      actor:NextImageSwipe 
        position:156,60
        wh:160,380 
        action:goto(Image3, push left)
    

Finally, the last image scene. It only has "backward" navigation in the photo set since we've only put three photos in our album:

scene:Image3
      image:image3.png
      actor:NavigateBack 
        position:14,33 
        wh:44,23 
        action:goto(Hawaii, push right)
      actor:PreviousImage 
        position:94,452 
        wh:23,17 
        action:goto(Image2, push right)
      actor:PreviousImageSwipe 
        position:0,60 
        wh:160,380 
        action:goto(Image2, push right)
    

We've just done simple scene-to-scene transitions in this Brief, but there are also several transitions you can perform directly on actors such as moving them, fading them in and out and others. I've created a syntax cheatsheet which you can use as a reference for your own Briefs.

Compiling the Brief

Once we have our photos, it's time to start writing our Brief. As mentioned earlier, the Briefs application consumes Plist files. However you can't just create one out of the box because it will only have image file references in it, but not the actual data. The reason we built the compact-briefs command earlier, was so that we could take our intermediate Briefs file (usually called a source brieflist) and embed the image data directly in the final product.

However I find editing XML tiresome, whether it's by hand or with a tool. Briefs provides an alternate syntax in which to write briefs which is transformed by a Ruby script into the intermediate Plist format. You can then compile the source brieflist to the final format using the compact-briefs command. The flow between formats and tools looks like this:

In the Briefs-util project (which you checked out earlier), you can find the Ruby script in BS/bs. I run these tools all at once in a script named compile:

#!/bin/bash
    ~/Development/Briefs-util/BS/bs photos.bs > photos-source.brieflist && \
        ~/bin/compact-briefs photos-source.brieflist ~/Development/Briefs/sample/photos.brieflist
    

Installing the Brief

The next step is to build the Briefs application with our newly-compiled Brief. Switch back to the Briefs Xcode project and choose the "Briefs" application target. If you haven't yet added the new brief to the Xcode project, right-click on the "My Briefs" and choose Add > Existing Files. Select the photos.brieflist file in the samples directory and in the following dialog make sure the checkbox at the top is unselected.

Now you can build the Briefs application for either the simulator or the device. If you're building for the device, make sure you have a proper provisioning profile in place that matches the bundle identifier of the project. Now we can Build & Run the project, and our photos.brieflist should appear in the list:

Select our new Brief and play with the application. To exit a Brief, touch and hold your finger down until a new screen appears asking you if you want to exit the current Brief.

Distributing the Brief

The idea behind these prototypes is putting a tangible example in front of users to get their feedback. So what do you do if you want to distribute Briefs to a wider audience? Ad-hoc builds are a pain and require access to the physical device. Fortunately, Briefs provides a tool called "Briefcasts", which is simply one or more Briefs embedded in an RSS feed. The Briefs application can retrieve Briefcasts via HTTP, so distributing Briefs is a snap if you have a web server to serve new Briefcasts from. This means you can install the Briefs application once on a device and provide updates anytime without having direct access to the hardware.

Currently, there isn't a tool to easily create a Briefcast directly out of a list of Briefs. For now, the project web site recommends copying and modifying the RSS template below:

<?xml version="1.0"?>
    <rss version="2.0">
      <channel>
        <title>*Briefcast Demo*</title>
        <link>*http://island.local/~alex/briefcasts/*</link>
        <description>*Demonstrate how awesome it is to use a briefcast to get briefs on the iPhone.*</description>
        <language>en-us</language>
        *Thu, 12 Nov 2009 03:05:00 GMT*</pubDate>
        <lastBuildDate>*Thu, 12 Nov 2009 03:05:00 GMT*</lastBuildDate>
        <item>
          <title>*Photos.app Sketch*</title>
          <enclosure url="*http://island.local/~alex/briefcasts/photos.brieflist*" length="*954708*" type="application/brief" />
          <description>*An example brief of the built-in Photos.app.*</description>
          *Sun, 13 Sep 2009 03:05:00 GMT*</pubDate>
          <guid>*http://island.local/~alex/cast/1#item1*</guid>
        </item>
      </channel>
    </rss>
    

The bits marked in bold need to replaced with your particular details. In the example above I only have one Brief, but if you had more than one, you would just include additional elements. You need to have this RSS file on disk as well as the brieflist file referred to in the tag.

Assuming that you have your web server all setup with the RSS file as well as the Briefs files, you can launch the application and add the Briefcast by selecting the "+" button in the top-right of the home screen. Enter the details on the next dialog and touch the "Save" button:

On the next screen you'll see the details of the Briefcast. Each Brief in the Briefcast will be listed under the section titled "Enclosed Briefs". Touch the "Download this Brief" button to retrieve the associated brief.

Once the brief is downloaded, you end up on a rather unhelpful confirmation screen:

You have to touch the screen, then navigate back to the top. Then you'll find the new Brief in your list of briefs. To update the Brief, you navigate back into the Briefcast section from the home page and download the correct Brief again.

One nice enhancement I'd love to see is a custom application URL for Briefs so that you could just email the link to someone. They could read the email from their device, select the URL and Briefs would launch and download the newest Briefs.

Conclusion

Briefs is a great tool for building live prototypes. Since it's a relatively young project, it still has a lot of rough edges, but I expect it to mature over time.

Don't hesitate to dive into the code behind Briefs. There are occasions when Briefs (the application or the command-line tool) will crash without much detail. I found that this is usually the result of bad syntax in the Briefs so a little sleuthing may be required to get things working.

Resources

Here's a list of related resources:

• Briefs Homepage
• Briefs on GitHub
• Sample Brief of Photos.app
• Briefs Syntax Cheatsheet
• iPhone Templates for OmniGraffle
• iPhone PSD for PhotoShop

LaunchBar

When I first switched to the Mac, I discovered Quicksilver and it was like discovering something essential that you didn't know you needed. QS was a revelation to me because it drastically reduced the mental distance between the what I wanted to accomplish and executing it.

Over time though, I found some things in QS that just didn't work consistently. It would crash and some of configuration was pretty clumsy. It became pretty clear that there was little, if any, forward progress being made on it. So, on the advice of several local Ruby/Mac nerds, I tried out LaunchBar. At first I was disappointed that it didn't have the breadth of QS, but after investing a little time and effort I found that it handled 99% of the things I wanted. More importantly, there was somebody actually working on it which meant it was going to evolve and improve.

Now I can hardly use a machine that doesn't have it installed. I can't believe people use the dock to launch apps. Yuck.

Mailplane

By and large, I'm biased to using the built-in Apple-given tools where possible. Apple usually gets it right and most other tools are designed to work with these apps. I've used Mail.app since I switched to the Mac, but always felt the interface to be inferior to GMail's tag-oriented structure. But since I had some non-GMail accounts, and I wanted to have a single app, I stuck with Mail.app.

However, once I dropped my last IMAP-only account I was ready to switch back. The only parts I really missed were the tight integration with the desktop that Mail.app brought. Lo and behold, along comes Mailplane to fill the gap. It is, essentially, a WebKit view of GMail with some extra bits to glue the web application to your desktop. I think it's brilliant and was happy to part with some hard-earned money for the privilege to use it.

zsh completion

My introduction to zsh came many years ago from a co-worker who boot-strapped my entrée into zsh-land with his configuration files. For the longest time I treated them like mystic, sacred tablets to be worshipped, but never to be understood. But over time I found little gaps I wanted to fill in with my zsh setup, especially with command auto-completion.

I'm a keyboard kind of guy. I like the command prompt. I also like going fast, so anything that makes me faster in the command prompt is like gold to me. I love shell auto-completion and found lots of things I wanted completion for, like RubyGems. So this year I finally pounded through enough documentation and samples to put together a few home-grown completion bits. I still don't understand most of the incantations that make it go (guilty as charged of cargo-culting) but it has made my command-line work-flow that much better.

Dropbox

Since switching to the Mac, I've always had at least two machines that I needed to share files between. For the longest time I used to store files in S3 then sync them back and forth. This worked, but it sucked because I had to do it manually and I had to get the order right so I didn't clobber any files.

When Dropbox came along, the clouds parted and the golden light of heaven shown down from above. Dropbox makes sharing files super-duper, idiotically simple. Combined with tools like 1Password, having consistent data across multiple machines is like magic on a daily basis—it just works and I don't even have to think about it.

One Bus Away

The thing I love about the iPhone is how it's an Swiss Army Knife for information in the modern age. You can find apps that do very specific things, very well and put them in a single box that you keep in your pocket. The One Bus Away app is one of my favorite iPhone tools.

I ride the bus a lot and to have real-time transit data in the palm of my hand is fantastic. No guessing about when the next bus is coming. No extra waiting. I know just when to leave to catch my next ride.

Mint.com

I was a long-time Quicken user but finally had it last January. It's a bloated app with serious usability issues. I finally reached a point where using Quicken had a net-negative impact on my life. It simply drove me crazy to use it.

Enter mint.com. At first I was very hesitant to give my information to these guys. But eventually I was convinced to try it because: * if I use Quicken another minute I'll go postal * sometimes the future doesn't start until you acknowledge it, and this looks like the future to me * I'm probably already more open to financial security risk anyway, so what the hell?

I get exactly what I want out of mint: a high-level dashboard that just updates itself. I use a few of the budgeting features which are nice, but really it's all about automated data acquisition and display. Oh yeah, did I mention that they have an iPhone app too?

Instapaper

Several years ago I built my own news-feed reader and one of the features I really wanted to implement was a temporary "holding pen" where I could put URLs to visit later. These were things that didn't quite warrant a Delicious bookmark, but I wanted to get back to.

By itself, Instapaper satisfied my original need and worked great as a bookmarklet in my browser. However it went from a convenient curiosity to an essential tool once other apps started integrating with it (e.g. Net News Wire, Tweetie 2) and Kindle integration (see below).

Oh yeah, did I mention that there's an iPhone app for that too?

Kindle (device and app)

The final tool of 2009 that made itself indispensable was the Kindle. It's a great example of "good enough". The Kindle is fraught with polish and usability issues, but they don't overwhelm the utility of the device.

I've been on a long dispense-with-all-physical-media kick, with books being my primary target. I have lots of books, but I don't have lots of space. Since I'm not emotionally wedded to the form-factor of paper books, the Kindle solves my reading-appetite vs. space problem quite well. I can stuff that thing full of books and not take up an inch of extra space in my house.

Like so many of the other tools I've listed, it's the other things that integrate with the Kindle that multiply its utility. First, the Kindle has gained a lot of adoption by other publishers. The Prags and O'Reilly, in particular, have been eager to jump into the e-book biz with first-class Kindle support. I probably go through about fifteen to twenty tech books a year and having them on the Kindle is great.

Another great bit of integration is how Instapaper delivers me a weekly digest of links I've captured right to my Kindle. I can read the full content of every page I bookmarked right there on my Kindle without the need for a network connection. This is a great for travel.

So?

Looking over this list, if there's one theme that emerges, it's all about ubiquity and convergence. (Did I really just say that?) Despite all of the queasy feelings those "market-tecture" terms give me, I don't quite know how else to define it. For me, 2009 was the year that I could get to my stuff whenever I liked, however I liked, through a multitude of channels.

I wonder what the theme of 2010 will be?

With the exception of Gehrig, I've purposely ignored anyone before the 50's. In my opinion, the game is just too different. I don't feel like I can trot Cy Young out as my starting pitcher just because of 511 wins. Those wins don't compare at all to wins in the 60's, 70's, 80's or 90's.

I'm also not penalizing anyone for being part of the Steroid Era or for even having used PEDs. There are, no doubt, some effects of these substances on the game, but I don't see the evidence being as conclusive as some would have you believe. Did Barry Bonds cheat? Yeah, I'm pretty sure he did. He was a helluva ball-player before he started and he still would have put together a Hall of Fame career without them. Steroids (a rather inaccurate term for PEDs) don't make you see the ball any better or improve your hand/eye coordination. At best they allow athletes to recover faster from working out. They don't build muscle, they amplify the body's own healing process in order to work out more. You can call steroid users cheaters, but I wouldn't call them lazy.

So this is my list. If you have a different one, I'd love to hear about it. So, without further ado, here's my list.

Catcher: Roy Campanella

Great all-around athlete, hit the crap out of the ball, threw runners out and worked well with pitchers. He was probably one of the most complete catchers to play in the majors.

First Base: Lou Gehrig

OK, I cheated a little bit here and dipped back in time further than I said I would go. But c'mon—it's Lou Gehrig.

Second Base: Jackie Robinson

They retired the number 42 for lots of reasons—those are the same reasons he's my starting second baseman.

Third Base: George Brett

Brett is the last guy to come close to hitting .400. He ended the 1980 season with a .390 average. What's even more impressive is that he had more home runs than strikeouts that year. He's got the longevity, the ring and the career to put him on my list.

Shortstop: Alex Rodriguez

Does A-Rod's admission to using PEDs damage is Hall of Fame credentials? Maybe, but as I contended earlier, I'm not convinced that they can carry a player to greatness. We love to dump on A-Rod's failures, but it's really only because he's been spectacularly successful in his career. Drugs or not, he's my starting shortstop. This, coming from a jilted M's fan too!

Left Field: Ted Williams

The Splendid Splinter. The greatest hitter of all time. Period.

Center Field: Willie Mays

Hard to find a better combination of offensive and defensive skills.

Right Field: Hank Aaron

How could I possibly leave Hank Aaron off the list?

Designated Hitter: Edgar Martinez

Edgar Martinez presents a serious conundrum to Hall of Fame voters. He's really the first "pure" designated hitter that voters will have to face. His career are numbers are impressive for the time he played, but he doesn't have some of the magic totals people like to see. That said, he's my vote for DH. Plus, I named my dog after him.

Starting Pitcher: Sandy Koufax

There's probably no roster spot that would engender more differences of opinion than starting pitcher. There are so many great ones to choose from. Koufax didn't have the career longevity like Satchel Paige or Tom Seaver, but at the height of his powers he was simply jaw-dropping amazing. His years from 1963 through 1966 are some of the sickest consecutive years a pitcher ever put together.

Relief Pitcher: Mariano Rivera

While there may be controversy over starters, the choice for reliever is pretty straight-forward. Seriously, is there any disagreement here?

Manager: Earl Weaver

He managed Hall of Fame players, he won a couple of rings, he wrote the seminal book on baseball strategy and he had a video game named after him. Yup, that fits my criteria.

I always wanted a personal knowledge base like this, but I couldn't do it the way she had. First, I have never had the space to collect that much paper. I cannot have, and do not want, a giant filing cabinet of folders. Second, I don't have the kind of recall she did to find that stuff. I could certainly remember that I once filed something, but wouldn't have a clue where I filed it. Which brings me to the third issue, which is suffering with the restriction of filing information under a single hierarchy.

When I first started using GMail I had a major epiphany about the differences between organizing information in folders versus tagging. These days that seems about as earth-shattering as realizing our galaxy is helio-centric, but it seemed like a pretty radical idea at the time. I flat-out love this style of organization. I may not remember where I filed something, but I'm pretty consistent when it comes to labeling things. With systems like GMail and Delicious, I just slap as many tags as I think are reasonable for a message or bookmark and I have a really good chance of finding it again when I need to.

But email and bookmarks aren't a knowledge base. For me, they are a reference, but usually aren't in a succinct enough format to be a good reference. I want to boil down newly-acquired knowledge into some quick, efficient prose. What I needed was a system that: * Works across multiple machines (e.g. work and personal) * Is fast to add and edit content * Is (relatively) searchable * Has some structure * …but not so much that I fight with it

So over the last few years I've started building a personal knowledge-base in earnest. It's not overly complicated or too involved. I simply keep a set of text files (specifically, files compatible with org-mode) in a special Dropbox folder. Dropbox makes synchronizing stupid-easy. I just treat these files just like local files and they appear on both my work and personal machines.

For a while I used VoodooPad, which is a very cool program. But I like having a little structure in my notes and imparting that structure in VoodooPad took more effort than I wanted. Later, I discovered Emacs' org-mode, which turned out to be a perfect fit for me—it's a relatively lightweight structure on top of simple text-files. The only thing org-mode doesn't do well is handling non-text media. I've considered moving to something like OmniOutliner, but I've found that org-mode works surprisingly well.

The technology isn't particularly interesting, nor is it what makes this work for me. Rather, it's the discipline of writing down little notes on various topics of interest. For example, I can never remember how to enable zombies for Cocoa debugging. I looked it up once, then added a section to my xcode.org file on NSZombieEnabled. I get two benefits out of this: first I've got it somewhere handy that I can easily look up again (via spotlight, grep/ack, whatever). Secondly, the very act of formulating a paragraph or two on the concept somehow reinforces it into my brain. In fact, it's very likely that I won't ever have to look up how to track zombies again.

The first, is a gem I started over a year ago, called word-salad. It uses the built-in dictionary file to randomly pull words out. This is great when you just need to generate a bunch of English-like text, but don't want to fall back on the boring old "lorem ipsum…" routine.

3.words ==> ['draw', 'ameliorate', 'bonanza']
    2.sentences ==> ['Shoot jonesing the make castle.', 'Blue murdered slight bastion.']
    2.paragraphs ==> ["Brachypterous gastropod pheretrer overeager toploftily denaturalization stokesite demented benzalhydrazine archaeologic. Haverer hypophonous lenticularly brickliner urocele paucipinnate pik unprintably perhalogen. Subglenoid bearish gesticulative staircase gallop vesuvianite pneumatically overyear conterminous dreamish. Nonalliterated galliwasp superconfirmation Comandra entoil millionth parcellize rarefaction Cynoidea. Podolian metamorphosable nativeness integriously protonematoid undoctor stochastically dissatisfactory unchastity.", "Increate unloquacious unsatisfiedly flareboard internuncio beguine equivocation snowshoe Rhynchonellacea. Parochially curliewurly vermix consistorial cond consciencelessness Anaxagorize recoct sempiternally Campanulatae. Scorpionida Castalides homoanisic semipenniform Novemberish assessor preterlethal acrotarsial knoller hartin. Procrastination boatwise canonize differentiate faunlike countermarriage obstinance dilatableness drumloid. Gerate squirr Silvanus Physostigma booting thyroarytenoid diminutival legpuller medisance radiobserver."]
    

The second is a gem for automatically retrieving and locally storing sales reports from the iTunes Connect site. If you have an iPhone app in the App Store, you already know how much (anti-)fun it is to get reports from Apple's site. So I put the itunes-connect gem together to help out. It's pretty basic for now, but will probably grow more capabilities as time goes on.

This all works great—until it doesn't. The ability to stage parts of a file is one of my favorite features of git, but there are times when you have changes that git won't let you split apart. Usually I give up at this point and just abort the rebase.

But yesterday I must have had too much coffee because I was determined to figure out how to do this. The changes that I wanted to split out were pretty easy to find. It was easy to remove them by hand and commit those. But I didn't want to have to manually add them back—that was too mistake-prone. What I needed was a way to invert the commit that removed those lines. What I needed was git revert. If I could revert my subtractions (in effect, adding the changes back), then I could squash the commits and rewrite history.

So here's what I did:

1. Started an interactive rebase session (git rebase -i)
2. Marked the commit I wanted to split as an "edit"
3. Manually removed the code I didn't want in the first commit, staged the changes and made a new commit
4. Revert the subtraction with git revert HEAD
5. Complete that rebase with git rebase --continue
6. Start a new rebase and squash the original commit with the manual changes one

Note that I didn't use git reset after step 2. It wouldn't have helped me because interactive staging wasn't working for me. I also marked all the commits in the second rebase as "edits", so I could fix the commit messages.

Put another way, the commits transformed like this, from left to right:

Maybe I'm overly fussy about clean commits. I'll admit that at times that my attention to cleanliness borders on the obsessive, but having the ability to go back and clean things up is one of my favorite things about git. Maybe I'm just not smart enough, but I usually don't know nearly as much at the beginning of a commit as I do at the end. With git, I can make it look like I do.

Changes:

1.0.2 / 2009-10-02

• Merged patches from Adam Salter:

  • Added new "presence" method for options.
  • Updated rspec usage to match the latest & greatest syntax
  • Make descriptions optional for flags and options

• http://clip.rubyforge.org

But now it's time for a change. My professional clock is set to about three years—when that bell rings it's time for me to move on. This was as true at Evri as it has been anywhere else, with one major difference: I made a career-change along the way.

OK, I haven't given up software to become a sous-chef or to take troubled kids out into the woods to teach them discipline. But I think I have reached the end of my professional Java programming career. This is kind of a "Big Deal", because I've been doing it for a decade. I know, I know: never say never. It's entirely possible that I'll eat my words a year or two down the road, but for now, I can't help but feel like I'm closing a major chapter in my professional life.

I've been with that language for a long time. Lately I've made a sport of complaining about it a lot, but honestly that's simply contempt bred from familiarity. There is a lot of good about the language and, despite my railings, the baby needn't be thrown out with the bath-water. What it comes down to is that I simply don't enjoy it anymore. It used to be fun to put those pieces together, but not any longer. So I'm trading in my Java hat for my two new loves: Ruby and Objective-C.

I may be stupid, but I'm not foolish enough to think that these languages will be the only remaining languages of my career. First of all, I'm not in the habit of defining my identity from the particular technologies I work with. Secondly, these are simply the environments I'm in love with now. I'm a dedicated family-man when it comes to people, but when it comes to programming languages, tools and environments I'm as bad as Liz Taylor. But jeez, why not embrace change? I don't do this for the paycheck. I do it because I love it. Why spend my time on things that aren't making me happy anymore?

With that question burning a hole in my brain, it became clear that, for now, the twisting road that is my professional career is going to take a long detour through Ruby and Objective-C. I hope it gets some time with Erlang too—I've only dabbled with it a little bit, but found myself quite attracted right away.

As I realized that my time was at an end at Evri, the question of "what next?" loomed large. Was there someone out there that wanted someone with the skills I wanted to cultivate? If so, were they doing something cool? Was I going to go "indie"? What would that look like? Questions, questions, questions. Fortunately I stumbled across a gig that is going to let me scratch most, if not all, of those itches.

I'm taking October off to continue and complete the series of iPhone screencasts I've been writing for PeepCode. In November I'll pick up my full-time gig and kick off the next phase of my professional life. You can't imagine how humbled and fortunate I feel to be able to do this.

I'm pleased to announce the release of the first in a series of PeepCode screencasts for the iPhone SDK I've been co-writing with Geoffrey Grosenbach. The first one (done in two parts) covers View Controller basics. We decided to start with View Controllers because they are at the core of all "productivity"-style applications. Once you have them down, you can start to put some meat on the bones of your iPhone apps.

Once we've covered those, we'll work our way through the entire MVC Holy Trinity, one bit at a time. Along the way we hope to deep-dive on a few ancillary topics like Core Location, Game Kit as well as provisioning and unit-testing. We also hope to look at visual technologies like Core Graphics and Core Animation, as well model-related APIs like Core Data and web service-related libraries.

The iPhone SDK is huge and there's a ton of material out there to cover. We're very excited to kick off what we hope will be a useful and successful instructional series for perspective iPhone developers out there. Let us know what you want to see.

I just wanted to make a quick announcement that the iPhone app I've been building at work just released a major update today. It's in the App Store and it's free. Check it out.

Working on this release was a ton of fun. We cleaned up a good bit of the mess from the earlier version and added some cool new features. What was even more fun was removing functionality that either didn't work well or wasn't obvious to the user. My god, there is nothing I love as much as deleting code. I don't know why I don't look for more opportunities to do this. Not only is it personally and emotionally liberating, it's often the "right" thing to do.

The features we pulled out really had no business being in there in the first place. Of course, we didn't know that until we released the first version, got feedback from folks and spent some time outside of the development process. Y'know, hindsight, 20/20, blah blah blah.

I think it is exceedingly difficult to get perspective on your work while you're creating it. There's just too much personal investment in the activity, especially if you get any sort of "flow" going. Creative types love "flow" and hate giving it up for any reason, even if the work suffers as a result. I think that's fine, but that's why there's a separate process known as "editing". The trick, of course, is to balance satisfying your possibly-self-indulgent creative urges with creating something of value.

Anyway, that's enough of my philosophical rambling. We also put together a little brochure site at http://evriverse.evri.com, including a screencast with music composed by yours truly. That process was a whitepaper-grade case study in "stuff you can do with iLife". I grabbed all the footage using iShow U, did all the music in Garage Band (on the bus using the computer keyboard no less) and glued it all together in iMovie. Super simple, super fun.

All in all I'm pretty pleased with the result. It's free, so check it out and let me (or Evri) know what you think.

In this post we'll look at three ways to allow your objects to communicate with each other in a highly-decoupled manner using the Cocoa frameworks. Cocoa provides three common ways of connecting objects for messaging: Notifications, Key-Value Observation and delegate pattern.

Notifications

Both Cocoa and Cocoa Touch provide a notifications framework built on two classes: NSNotificationCenter and NSNotification. The former contains a singleton instance, (via the defaultCenter method) with which observers register themselves and observables post notifications. The latter is the "envelope" for the notification which can contain an NSDictionary instance of customized data as its payload.

When an object registers for notifications, it specifies a notification to listen for (as a NSString) and, optionally, a target object. If the target is set to an object, only notifications posted by that object will be received. If the target object is set to nil, any object posting that notification will be sent to the receiver. Senders can optionally include a NSDictionary instance filled with domain-specific objects. You can use this as a way to further parameterize a notification, or eschew it altogether.

Consider the example below:

@implementation Foo
    - (void)init {
      if (self = [super init]) {
        [[NSNotificationCenter defaultCenter] addObserver:self
                                                 selector:@selector(didReceiveNote:)
                                                     name:@"TheBigNotification"
                                                   object:nil];
      }
      return self;
    }
    
    - (void)didReceiveNote:(NSNotification *)notification {
      NSLog(@"Hey, I got a notification with user info: %@", [notification userInfo]);
    }
    @end
    
    @implementation Bar
    - (void)doSomething {
      NSLog(@"I'm up to something!");
      NSDictionary *stuff = [NSDictionary dictionaryWithObjectsAndKeys:@"foo", @"bar", nil];
      [[NSNotificationCenter defaultCenter] postNotificationName:@"TheBigNotification"
                                                          object:nil
                                                        userInfo:stuff];
    }
    @end
    

Here the Foo class registers itself as a notification receiver for the notification named "TheBigNotification" on any object. The Bar class will post the same notification when the doSomething method is invoked.

We could have been much more specific about which object to observe, but I think this violates one of the key features of notifications which is that the sender and receiver are decoupled. To me if the receiver is going to go to the trouble of listening for notifications from a specific object, you'd be better off declaring a custom protocol and using the delegate pattern (explained below).

While at the 2009 WWDC, I had a chance to pick the brains of some Apple engineers and one of them told me that notifications are really intended for one-to-many broadcasts. I think that's a great rule of thumb, because the code can really feel like overkill for single object-to-object messages. However, I'd add one more clause to that rule which is that notifications work for one-to-one messages when you would otherwise have to pass instances around of delegates just to connect objects.

As an example, consider an iPhone application with a stack of view controllers in a UINavigationController instance. If you have a view controller several steps removed from another view controller (i.e. one is a further up the stack in a tab bar controller) and it needs to call some method back to the view controller lower down in the stack, each intermediate object would need a reference to the delegate class just to pass it down the line. This seems like the worst kind of encapsulation breakage since you're forcing a bunch of otherwise-unrelated classes to have knowledge of a class they don't even use.

Maybe it's my long track-record with Java, but I pay attention to the classes I import and I like to keep that list as short as possible. The more classes know about each other, the more difficult it is to modify them.

There's one final thing worth knowing about the NSNotificationCenter, and that is how it works with threads. When one object posts a notification, that call blocks while the notification is delivered to all targets. This means that you want your notification-handling code in the receiving object to be as quick as possible. If you can live with deferred notifications and want to avoid the synchronous nature of notification delivery, you can use the NSNotificationQueue instead.

Key-Value Observation

When I first read about, and used Key-Value Observation (KVO) it seemed like magic. You simply point one object at another and say "I'd like to know when this attribute changes" and Cocoa handles all of the notifications and threading automatically. Genius!

KVO also allows you to use a special syntax when specifying the attributes you want to observe so that you can register with an object and traverse its object graph. Let's look at the classic customer/orders/line items example. If you want to know when changes are made to any line items in an order you would observe changes in the order like so:

Order *order = [self anyOrder];
    [order addObserver:self
            forKeyPath:@"lineItems"
               options:NSKeyValueObservingOptionNew | NSKeyValueObservingOptionOld
               context:nil];
    

Whenever you register for KVO, you must implement the method - observeValueForKeyPath:ofObject:change:context:. Unlike notifications, you don't get to specify a selector. This means that if you're object is observing several other objects, you will have to inspect the object given in the callback method and dispatch appropriately.

Key paths have a ton of flexibility including the ability to traverse deep object graphs and observe aggregate functions on a collection (e.g. observe the max date of all of a customer's orders). Check out the Key-Value Programming Guide for details, which is an indispensable reference.

KVO works like notifications in that the thread that invokes the change will block until all observers have been notified and their callback methods have been invoked. Unlike notifications, there is no built-in asynchronous KVO-triggering mechanism unless you explicitly mutate properties in another thread using something like the detachNewThreadSelector:toTarget:withObject: class method of NSThread.

For classes to be key-value observable they must be, what the documentation calls, "key-value compliant". This generally means that each observable property exposes certain methods for accessing and mutating them. Again, the KVO guide is your go-to reference.

KVO is structurally similar to notifications, but is really intended for fine-grained messaging, generally around your model classes. Notifications are intended for broader application-level events. Both mechanisms give you a way to decouple classes that would otherwise need more intimate knowledge of one another.

Consider the simple case of managing a tabular view of line items in an order. When a user adds a line item, removes a line item or updates the quantity of a line item we want a field showing the total price to be updated. Without KVO we would need to add extra logic in our event-handing for the table view to also update the total field. With KVO our event-handling logic can simply focus on updating the underlying model. The total field will simply observe the total cost of the order and be updated appropriately. Now if we want to remove the total field or perhaps add another view of total information, we don't have to touch the original event-handing code.

Delegates

If you've spent any time with the Cocoa docs you'll run across The Delegate Pattern. This thing is like the quark—it is the fundamental building block of the Cocoa APIs. Conceptually, it's really pretty simple. A delegate is simply an object that provides custom behavior for another object, often by implementing a specific protocol.

Let's return to our running example. A table view of line items in an order has to handle a lot of things such as rendering each cell, handling scrolling, managing user-generated events and so on. These features are common enough that they are simply part of the table view class itself. What is specific to your application is the data that goes in those cells and the actions to be taken for user-generated events. Cocoa solves this by providing protocols for delegating that behavior to custom classes. In the case of providing data, a data-oriented delegate would be queried by the table view for the total number of rows, appropriate object to put in each row, the column headers, etc. In the case of event-handling, the table view will forward events to the event-handling delegate.

Notifications use the NSNotificationCenter (or by proxy, the NSNotificationQueue) as an intermediary between message-senders and message-receivers. KVO uses the observed object as the intermediary. In the delegate model there is no intermediate object, but you still have relatively decoupled code because the object that invokes methods on the delegate has no knowledge of what class of object it's dealing with. Put another way, you could remove all of the classes in your project that implement a particular delegate protocol, and the class that uses the delegate would still compile correctly.

The delegate pattern isn't so much about decoupling which objects are communicating with each other, as much as decoupling the classes of those objects. This doesn't make it any less powerful than the other two methods. In a lot of cases a simple delegate model is much more straightforward and preferable the alternatives.

The delegate pattern is Cocoa's version of dependency injection. The object receiving the injected dependency has no idea of the type (and by extension, the implementation) of the object it is receiving, only what it's capabilities are via a contract specified as a protocol. This is basic polymorphism where the interface and the implementation are separated to reduce coupling.

Delegating is a popular alternative to sub-classing. It is a preferred alternative because it's less likely to break encapsulation. If customization is done through subclassing it can be difficult to know which methods to override and implement, and easy to break the parent class. This is why Java provides all sorts of safety features like marking methods abstract and final. However that is simply more of a headache to deal with and using polymorphism would be a much better design.

So there you have it, three ways to keep your code flexible and loosely-coupled!

Resources

• Apple's Notification Programming Topics
• Apple's Key-Value Coding Programming Guide
• Apple's Cocoa Fundamentals
• Mike Ash's Well-Reasoned Critique of KVO Design

We live in a very interesting time for application development. The distinctions between desktop, browser and mobile applications are blurring more and more every day. "Ubiquitous platform" applications, like Evernote, are where the future is because it reduces the major hurdle of access for users. As time goes on users are going to expect applications to be available in a bunch of different ways that all need to work together.

The common way to do this is base the application on shared-state accessed via an HTTP-based API. This makes it relatively easy to create apps for various devices as well as offer up a public API for others to use. An interesting side-effect of this is that the API, the shared-state and data behind it become the differentiator between products. The end-user client software is simply the packaging, turning the stand-alone application market into something more akin to the "razors and razor-blades model".

Okay, none of this should be earth-shattering news to anyone. You should be prepared to write more apps that are integrated with one (or possibly more) "backend services". These services will, as likely as not, be connected over HTTP. As an iPhone developer you should know how to do this in your sleep.

The Evri API is absolutely crucial to the EvriVerse phone application. Without it, the app does nothing of interest on its own. Our iPhone application is merely one kind of presentation of our web APIs. I don't think that's unique. What good would Tweetie be without Twitter?

Dealing with Web APIs

An unfortunate fact of integrating with web APIs is that they are, relatively speaking, slow. We don't want the user to get frustrated and give up on our app because they have their flow continually disrupted by waiting for data.

There are some things you can do to mitigate the latency costs, if you control the API such as using some sensible caching parameters. However, especially if you don't control the web API, the bulk of what you should focus on is concurrency within your application. The biggest arrow in your quiver is figuring out how to turn unnecessarily serial operations into parallel ones. Now keep in mind that the iPhone doesn't support true multi-core concurrency. What I'm talking about is taking advantage of the I/O wait times that are inherently part of working with a high-latency I/O source (the web).

Coming from a Java background, using threads seemed like a pretty natural approach and after reading the docs on the NSThreadclass I felt pretty confident I knew how to make it work. However, I was surprised by how quickly the code complicated. There is a lot to like about Objective-C, but I sorely miss Ruby's blocks and even Java's inner-classes. In Objective-C you connect methods together dynamically by handing objects selectors. The problem with this is that it's difficult to tell the high-level methods from the low-level callback methods just by looking at your class definition. Yes, you can use special comments and the like, but if you've worked in a language that has structural support for this differentiation, moving to a language without it feels like losing an arm.

So instead of managing your own threads, let Cocoa Touch do the work for you. The NSURLConnection class is inherently asynchronous—you can fire a request and immediately move one with your life. Now you just need a little structure for handling the response and getting it back into your views.

The EvriApi

We tried to push as much of the mechanics of URL-handling into a separate code-base we call EvriApi. A good bit of the design was inspired by Matt Gemmell's MGTwitterEngine, so I can't claim that I came up with the whole idea myself.

We wanted the API to be as simple to work with as possible. Each API call has the same structure: * Each API method takes zero-or-more domain-specific parameters, a selector and a target object. * Each API method returns a unique request identifier * The target object will have the given selector called when the request completes, and will be given an instance of the EvriAPIResponse class. * If the response is successful (check [response success]), the payload of the request is retrieved as a domain-object * If the response is unsuccessful, check the error message on the response * Requests can be cancelled by handing the request identifier back to the API * Attempts to cancel already-completed requests do nothing

The NSURLConnection class makes asynchronous requests by default, and uses the familiar delegate model for handling the response. When a client invokes an API method, it returns immediately with a unique identifier (as a NSString instance) and the request is initiated on another thread which is handled automatically by NSURLConnection. The client is responsible for keeping track of that and using it to cancel any outstanding requests.

When the request completes, the body of the response is parsed using the NSXMLParser class. The parser is given one of our specialized XML handlers as a delegate. These handlers deal with the XML event-stream and produce a model object or collection of model objects. The framework then packages this up into an EvriAPIResponse instance and invokes the selector given in the original request on the target given in the original request.

Wiring it Up

The EvriVerse app is primarily a "productivity" style application. In Interface Guidelines-parlance this means that we use a lot of hierarchical navigation. In general we have no more than one or two API calls associated with a particular view controller.

The common pattern for handling a user-initiated action that requires an API call is:

1. A new view-controller is instantiated and displayed, typically with an activity indicator visible
2. User-interaction is disabled on the new view controller
3. The new view controller is displayed (usually by pushing it on the navigation controller stack).
4. A request is submitted to the EvriApi, setting the new view controller as the target and some special method as the target selector.
5. The request ID from the EvriApi is given to the newly-created view-controller
6. When the API request finishes, the handler method on the new view-controller executes, handling both success and failure cases. It also re-enabled user interaction on the view.
7. If the user navigates back before the request has completed, any outstanding requests are cancelled. This is usually handled in the viewWillDisapper:(BOOL)animated method of the view controller.

Stepping back, the actors and interactions look something like this:

In code (snippets) it might look something like this:

    // our parent view controller
        @implementation ParentViewController
        - (void)didSelectEntity {
          ChildViewController *cvc = [[[ChildViewController alloc] init] autorelease];
        
          [self.navigationController pushController:cvc animated:NO];
        
          NSString *requestId = [EvriApi fetchEntityByURI:entity.uri
                                          performSelector:@(didReceiveEntityResponse:)
                                                 onTarget:cvc];
        }
        @end
    
    
    
        @interface ChildViewController : UITableViewController {
          NSString *requestId;
        }
        
        @property (nonatomic, retain) NSString *requestId;
        @end
        
        @implementation ChildViewController
        
        @synthesize requestId;
        
        - (void)didReceiveEntityResponse:(EvriAPIResponse *)response {
          // TODO: hide the activity indicator
          // TODO: re-enable user interaction
        
          if ([response success]) {
            Entity *e = (Entity *)[success result];
             // TODO: redisplay as necessary
          }
          else {
            // TODO: show an alert
          }
        }
        
        - (void)viewWillDisappear {
          [EvriApi cancelRequest:requestId];
        }
        
        @end
    

So there you have it, one developer's way of integrating the iPhone with web APIs. More to come. Stay tuned.

Ever since I saw Laurent's and Rich Kilmer's presentations about MacRuby and HotCocoa at RubyConf '08, I've been jazzed about what MacRuby brings to the world of Cocoa development. So check out the screencast for the coolest new Ruby on the block.

I wanted to kick this off with a little appetizer. We're going to look at how we implemented incremental find in EvriVerse. Evri has a huge database of structured entity data (people, places and things) and our users want to be able to search for any of them. After putting the initial prototype for EvriVerse on a few folks' phones, a lot of them wanted a search feature that worked more like our website.

In the original implementation a user would type in the search field, then hit the "Search" button and then get their results back in the table view. This takes too many steps, and we wanted something that felt faster and more responsive.

So our incremental find solution needed the following properties:

• It should only send a search request when there is a measurable pause in input
• It should enable some kind of activity indicator while a search request is running
• A user should be able to modify the contents of the search field while a search request is in-progress
• It should not preclude the existing two-step search process
• If a search request is in process when another one starts, the first one should be cancelled and the new one should be the only request in-process

At first I tried managing my own NSThread instance while tracking the timestamps of touch events in the searchBar:textDidChange: method (part of the UISearchBarDelegate protocol). If the the gap between time samples was big enough I'd cancel any outstanding request and start a new one. This seemed like a good approach at the whiteboard, but really fell apart in the implementation. I had race-conditions left and right and I had the sneaking suspicion that I was swimming upstream against "the Cocoa way".

As an aside, if there's one thing I've learned in my brief time with Cocoa, it's that mucking about with your own threads is a pretty crummy way to do things asynchronously. Usually there's a much more elegant, built-in solution in the Cocoa APIs that will handle the threading for you. I'll riff on this theme in a later post.

So back to the problem at hand. What to do? When I get stuck like this I often start thinking of how I'd do this in another language. This got me to thinking about how one would go about implementing this sort of thing in Javascript. This is standard Web-2.0 kind of stuff—right after drop-shadows and rounded corners, you learn about type-ahead search at AJAX School.

See, Javascript has this very nifty little function named setTimeout where you hand it a function and tell it how long you want to wait until that function is executed, and it returns you a reference identifier. There is a companion function, clearTimeout that allows you to cancel any previously-created "timeout" function by handing it the reference identifier you got earlier from setTimeout.

In the common AJAX-ified dynamic search that you often see, an event-handler observes the search field in a web form for key presses. Each key press cancels any existing timeout function, and starts a new one that will execute in whatever time-interval was chosen as the "quiescence period". The trick is that because you're scheduling future tasks, and you cancel any outstanding ones when you kick off a new one, you get the "measurable pause" feature in a simple, elegant solution.

Cocoa provides a very similar mechanism, through the NSTimer class. In my view-controller, the code ended up looking something like this:

@implementation FindViewController
    // much implementation code has been elided for demonstration purposes
    
    // observe text-field changes and fire the scheduled task
    - (void)searchBar:(UISearchBar *)searchBar textDidChange:(NSString *)searchText {
      if ([searchText length] > 2) {
        if (timer) {
          [timer invalidate];
          self.timer = nil;
        }
        NSDictionary *info = [NSDictionary dictionaryWithObjectsAndKeys:searchText, 
                              @"Text", 
                              nil];
        self.timer = [NSTimer scheduledTimerWithTimeInterval:1
                                                      target:self
                                                    selector:@selector(submitSearchRequest:)
                                                    userInfo:info
                                                     repeats:NO];
      }
    }
    
    // The method that runs in the NSTimer--equivalent to the function 
    // we'd pass to Javascript's "setTimeout" function
    - (void)submitSearchRequest:(NSTimer *)aTimer {
      NSDictionary *info = [aTimer userInfo];
      NSString *text = [info objectForKey:@"Text"];
      [activityIndicator startAnimating];
      self.requestId = [EvriApi fetchEntitiesMatchingPrefix:text
                                            performSelector:@selector(didReceiveEntitiesDynamically:)
                                                   onTarget:self];
    }
    
    // The callback from our API class, included for completeness
    - (void)didReceiveEntitiesDynamically:(EvriApiResponse *)response {
      [timer invalidate];
      self.timer = nil;
      [activityIndicator stopAnimating];
      if ([response success]) {
        [entities removeAllObjects];
        [entities addObjectsFromArray:(NSArray *)[response responseObject]];
        [tableView reloadData];
      }
      else {
        UIAlertView *error = [[UIAlertView alloc] 
                              initWithTitle:@"Uh oh…"
                              message:@"Sorry, we were unable to execute your request. Please try again.
    "
                              delegate:nil
                              cancelButtonTitle:@"OK"
                              otherButtonTitles:nil];
        [error show];
        [error release];
      }
    }
    
    @end
    

In the searchBar:textDidChange: method we check to see if our instance field, timer, points at an existing NSTimer instance. If it does we cancel it, clear the reference, and start a new timer instance. That timer is configured to run in one second and will execute our submitSearchRequest: method. This makes a call to our API and designates the didReceiveEntitiesDynamically: method as the callback for that asynchronous operation.

Finally, the didReceivedEntitiesDynamically: method handles the UI by disabling the wait indicator, dealing with any error cases and populating our table view if our request was successful. I'll write another post later about how the EvriApi class works and its design.

So that's it. Simple and elegant. One of the things I love about working on several different programming languages is figuring out how to apply the best tricks of one language to another.

That's all for now, but stay tuned! More iPhone posts to come…

After several months of work, I'm pleased to announce the release of Evri's EvriVerse for the iPhone. We've tried to capture the unique things we do at Evri into the awesomeness that is the iPhone. I won't spill all the details here, so check out out the "official" company post.

I've been working on this thing for a few months, mostly on a part-time basis. We have a 20% time program at Evri where devs can pitch ideas they want to spend one day a week on. I was dying to write an iPhone app and jumped at the opportunity to turn it into a 20% project. After some early prototypes I got enough attention that I was given a solid two-week sprint to finish it off.

This was a ton of fun to work on and I have five or six posts on some things I discovered while working on it.

It's free in the App Store now. We'd love to hear any feedback you may have, which you can post on the Evri Support Page.

For most of us at work, our primary Git experience is with GitHub. This is a good way to learn how Git's distributed model works, but GitHub provides some nice infrastructure that you simply don't have with a stock Git install. You have to cobble-together the rest from a variety of sources. So GitHub guys—if you're listening—it would be fantastic if you provided real white-label GitHub solution for organizations to install locally. You guys have really spoiled us. (oops, nevermind. Checkout GitHub:fi). But until then, we have to roll our own…

The Goals

First, let's review what exactly we were trying to accomplish. We wanted a solution that: * easily converted existing Subversion repositories to Git * provided a usable web view over all repos that was equal to, or exceeded, the stock SVN/DAV model * allowed us to designate a set of "authoritative" repositories * provided for "developer" repositories to allow developers to publish their changes * allowed for patch reviews (if teams decide to do that)

The Solution

Right away we knew that we wanted two different types of repositories: authoritative and developer. Authoritative repos are ones that host the "official" source code and from which we create release builds from. When a developer wants to start working on a project, these are the repos they clone to get started.

The other type of repository belongs to an individual developer. Developers have read/write access to their own repos, but others only have read-only access. This allows developers to share changes in a pull-request model similar to how GitHub works.

Permissions & Connectivity

Developers connect to these repositories in two different ways. We use the built-in git:// protocol (with git-daemon) for read-only access and ssh:// for read-write access. Developers clone authoritative repos and other developers' repos using the read-only git:// protocol. Developers have a remote reference to their developer repository using ssh.

For each project, we designated one or more maintainers that have write-access to the authoritative repositories. These folks accept patches from other developers, test and integrate them on their local machines, and push final changes to the authoritative repositories. We manage the authoritative repos using gitosis. Check out this great tutorial for details on setting up gitosis.

gitosis

On our central "git box" we create a "git" user which owns both the gitosis-admin repository as well as all the authoritative repositories. Gitosis is bootstrapped with a single SSH public key that gives the associated user the ability to write to that Git repo over SSH. That user can then add more keys and configuration allowing other users to manage projects. Gitosis works a bit like the CVSROOT project in CVS—it is a special Git repo that allows us to configure other Git repositories. We use gitosis to configure our designated maintainers for each project.

Our authoritative repos are housed in /home/git/repositories. Our developer repositories are located in /opt/repos, where each developer has their own directory to put Git repos in.

git daemon

We run git daemon to allow for anonymous read-only access. We configured git daemon to serve up all Git repositories found in the /opt/repos directory with read-only access. We also symlink the authoritative repositories (/home/git/repositories) into this directory so that we can serve up read-only access to authoritative repositories also.

gitweb

For all the power Git has, there sure are a lot of half-baked web interfaces for it. We did our best to vet the tools listed on the Git site. A good number of the them were either abandoned or simply didn't work. In the end we went with the venerable old gitweb.

Gitweb has a ton of functionality—including all kinds of search capability, different views (by tag, by commit, by tree) and even serves up an Atom feed of changes—but it is definitely a designed-by-developers-for-developers. (NB: In case you missed the markup, that is considered an epithet on my team).

Frankly I was amazed that somebody hadn't built a Rails app to manage this stuff. Yes, we looked at gitorious and even tried to set it up. After three hours I still didn't have a working installation and my patience had run out. While gitorious appears to be a nice turn-key solution, I don't think it's actually that great of a fit for us. So gitweb it is. Sigh.

The Flow

With me so far? Maybe not. Let's look at a picture then…

In this picture the authoritative and developer repos are drawn in separate boxes (since they're logically separate), but they are located on the same machine in reality. Let's cover a couple of common scenarios:

New Developer, Old Project

A developer starts by cloning an authoritative repo using the git:// protocol (remember, read-only). This will create, by default, a remote reference named origin that points back to the authoritative repo.

The next step is for them to create a developer repo in their directory for the same project on the central git machine. They also want to a remote reference to their developer repo so that they can push changes to that. That would looks something like:

git remote add alex ssh://git/opt/repos/alex/circus/clown-car
    

If this user is also a maintainer, they need to add a third remote reference which gives them read/write access to the authoritative repository. The way that gitosis works is by accepting SSH keys on behalf of the git user. So if you're properly configured, you can push changes over SSH by logging in as the git user. A maintainer would add a read/write reference like so:

git remote add main ssh://git@git/home/git/repositories/circus/clown-car
    

Because of the way gitosis configures the SSH keys in /home/git/.ssh/authorized_keys and the fact that the default protocol is ssh, this remote reference could also be added like this:

git remote add main git@git:circus/clown-car
    

Now because there are number of steps involved here, we wrote a little command-line tool (as a RubyGem) that takes care of these steps all in one go. Right now, it's very specific to our setup at Evri, but I can see how we might extract a common tool (oh boy…another side-project…)

Old Developer, New Project

When it's time to create a new project, a developer usually starts out by creating a new Git repo on their local box while they're building out the initial version. Once it's time to share, that developer can create a new developer repo on our central Git machine just by SSH'ing in and making the appropriate directory, adding the remote ref to their local repo and pushing changes. Again, our internal tools sets this up all in one go.

Once that project is ready to have an authoritative repo, a gitosis-enabled user will pull the latest changes for the gitosis-admin project. They'll edit the gitosis.conf file to add the new project (and members) and commit the changes. Then the developer adds a new remote ref (a read/write one as the git user) and pushes changes out to the main repo.

Sharing Patches

The most common flow is when developers share patches with each other. There are a couple of ways to do it. Developers keep their developer repository up-to-date and email requests to their teammates to pull changes from their repo (a la GitHub). Developers may also choose to share patches via email using git format-patch, git send-mail and git am.

Ultimately the maintainers are responsible for gathering patches from developers and integrating them back into the authoritative repositories.

For folks used to version control systems like CVS or Subversion this feels like an awful lot of hoop-jumping. One part that I can't diagram or explain as a series of technical bullet-points is the stewardship the pro-Git folks have to take on. People who are switching to Git get frustrated by the byzantine nomenclature and steep learning curve, so a big part of the change and setup is simply helping people out when they get stuck.

While doing Java development, I've often used Eclipse's file history feature to roll things back. The downside is that it's file-by-file and not easy to tag the current state of your entire project in one go. It appears that Xcode has this feature, but since I'm using Git for my SCM, I figured why not use that instead?

So my new flow has been to stage changes to the index whenever I make any progress and the app is still in a working state. This is different from a commit, which I still like to think of as a succinct, whole change around a particular feature. The incremental staging is more like dribbling micro-changes to the next stage prior to committing. It can take several attempts to get to a real, first-class commit.

I like using magit, so I keep Emacs running in the background. When I get to a good checkpoint, I stage hunks or entire files to the index. When I get enough to make a full commit, I commit them. If you're running git exclusively on the command-line, this would be the equivalent of using git add -i.

When I first started using Git, I couldn't see the value in Git's stage-commit-push workflow. Now I get it. Like most Git tricks, this one is probably obvious to a lot of folks, but for me it's been a real life-saver on this project. I've been able to be much more cavalier with experimentation because I can easily revert changes with a single keystroke. Nifty!

As anyone who has read this blog recently knows, the luster is coming off of Java for me in a big way. These days my goals are simply to co-exist with it in a way that keeps me happy. I'm not going to be able to toss Java overboard so my working-life becomes a question of how to be happy with the situation.

While our home-grown framework has worked well for us, I'd much rather see us using something with wider adoption and use. So I took a look at Jersey and came away pretty impressed. Looking at how the API is built and what you need to do to build REST-ful web services in Java I couldn't help but feel like Java has learned some lessons from the rest of the world.

The original Servlet API is what could be termed a "classic" Java API, wherein consistency and type-safety are the rules. In the Servlet API, you interact with requests and responses as monolithic objects. Each type of HTTP method (i.e. GET, POST, PUT and DELETE) has their own method which takes a ServletRequest and a ServletResponse object.

It's a reasonable first solution, but it breaks down quickly. The primary issue is that by packing all possible request-related data into a single request object, and all possible response-related data into a single response object the classes themselves start to feel bloated. A more sinister second-order effect is that these classes, especially the response object, have some hidden state and "gotchas" because they are so general.

As an example, consider the case where you want to return a response with a non-200 response code, an entity body and some headers. The order in which you set these on the response object is important because of how it's implemented. You need to set the status and headers before you write any output to the stream. This make sense because you want a stream-oriented interface which means not storing the entire response in memory and then flushing. But, due to the strict order of HTTP response messages, you need to flush the header information prior to sending the body. It's not a particularly difficult thing to remember, but it is unnecessary mental overhead that is an artifact of the implementation.

The request object is its own special brand of fun to deal with. Again its broad coverage makes for a rather clunky API to deal with. You want request parameters? You have to grovel through String arrays if you want to capture all of them. When implementing a Servlet, often what you want is some combination of request parameters, cookies and header; rarely do you need all of them at once. However what you get is one big über-object that has everything. Enjoy!

One final beef with the ServletRequest class is that getting path parameters out is an absolute nightmare. If you're building REST resources you really really care about the path as it is the way to identify resources. The poor support the ServletRequest class provides for this is simply shocking. Here's a String — you parse it and figure out what the hell the segments are.

In contrast, Jersey has a much looser philosophy with how requests and responses are handled. First, the monolithic request and response objects go away. Instead your methods provide the narrowest possible interface, expressing only what they need in exacting terms. This is done by making extensive use of Java annotations to mark up simple method parameters. For example, if you have a resource that needs a request parameter, use the @QueryParam annotation. Need a header? Just use @HeaderParam. Interested in cookies? Use the @CookieParam annotation. Here's an example:

public MyResult getMyResult(@QueryParam("name") String name) {
        ...
    }
    

This does away with a tremendous amount of "busy-work". You want the "name" parameter? By god you're going to get it—no intermediate objects to reach into and pull stuff out of.

A really nice side-effect of this is that writing tests for your resources become so much cleaner than using the more general Servlet API. Instead of setting up the monolithic request and response objects, you simply pass in the bits you want.

What about the response side? I think that it's in this area that we really see a fundamental philosophical shift emerge. In an earlier time the textbook answer to how to design an API like this would be to have each request method return some kind of superclass or interface. This would allow us to use polymorphism to vary the response, but keep our type-safety.

Jersey takes a different approach based on real-world needs. Whereas the Servlet API is intended to keep everyone equally happy by making sure everyone suffers the same amount of pain, Jersey lets you vary what you return—no special markers, no special configuration. If you have a simple case where you want to serialize an object as the entity response, just return the object.

What if you need to fiddle with the response some more? Maybe set some headers or alter the status code? In this case your method returns a Response instance. Now this may sound monolithic and perhaps, under the covers, it is. What saves it from degenerating into the Servlet API is the fact that you build a Response object with only as much as you need.

In the Servlet API you might have to set a moderately complicated response like this:

public void doGet(HttpServletRequest req, HttpServletResponse resp) {
      // do some stuff
      resp.addHeader("Expires", computeExpires());
      resp.setStatus(201);
      writeResponse(resp.getWriter(), new MyStuff());
    }
    
    private void writeResponse(Writer writer, MyStuff stuff) {
      // do whatever you have to do to serialize your object
    }
    

In Jersey it looks like this:

public Response getMyStuff() {
      MyStuff stuff = new MyStuff();
      return Response.created(stuff).expires(computeExpires()).build();
    }
    

The amount of code probably comes out to be the same, but the fact that you can build the response in a single line feels really good to me. The amount of vertical space dedicated to response-building is much more proportional to it's conceptual space in the method in Jersey than the Servlet API.

Now I realize that comparing Jersey to the Servlet API is perhaps unfair. The Servlet API is really designed to operate a layer below where Jersey plays. But I think the same arguments apply to a majority of the other popular web frameworks. The models are all essentially the same.

So here's where the philosophical sea-change comes in. To vary the response type to do the right thing means that after a decade of existence, people are finally embracing Java's reflection capabilities. Up until pretty recently this was considered the domain of the lunatic fringe.

The primary argument against reflection has always been that it's too slow. While it's true that their can be a penalty under certain circumstances, the modern perspective is that it's an acceptable price to pay for a kinder, gentler API. Simply put, this level of dynamism is essential for reducing the amount of boilerplate and busywork.

I think if other frameworks like Rails, Django and Merb hadn't gained so much traction in the last few years Java would have continued on it merry way. However I think the popularity of those alternatives has forced the Java greybeards to learn a thing or two. Strict static type-safety can become a burden and using reflection to vary behavior instead of polymorphism can make for some very clean APIs.

After spending a few days with Jersey I find myself a little re-energized to work with Java again. We're even considering Groovy as a way to cut down Java's chubby syntax for another small productivity win. Of course, my personal preference would be to do this JRuby since I, personally, don't find Groovy to be terribly compelling on its own. But since Jersey is so dependent on annotations, using JRuby is a non-starter.

Who knows? This could be a complete fiasco and another frustrated attempt at tilting windmills. I'm already a little suspicious of the "enterprisey" way this thing gets configured. However if nothing else, it's refreshing to see some new ideas make their way into Java after all these years.

Now you may ask yourself, "for the love of God, why would you want to do something like this?" Well, at work we do a lot of internal web services. For my particular team, we've found a real sweet-spot by using an embedded Jetty server sitting right next to a BDB instance. There are no extra processes or packages to manage (e.g. apache or a RDBMS). However we were becoming dissatisfied with our current web layer which is a homegrown REST framework that sits on top of the Servlet API. So in a fit of rage, I decided to see if I could stuff Merb in the middle of this mess.

You may also be asking yourself, "why not use the jruby-rack gem directly?" The answer is that the jruby-rack gem makes a lot of assumptions about how you want to run your application. First it assumes that you're cool with packaging things up as a WAR (which I'm not) and, secondly, that your application is primarily a Rails/Merb application. In my case, for better or worse, our app is really a BDB application with a Merb app glommed onto the side for web visibility.

The Solution

I can't take complete credit for this solution. If I hadn't found Jan Berkel's post on putting Rails in Jetty I would have never figured out how to stuff Merb in there. To give yourself some context, take a look at that post first. Then take a look at the "Merb-ified" version of the same recipe below. Both solutions assume that you're configuring Jetty within JRuby.

server = org.mortbay.jetty.Server.new
    thread_pool = org.mortbay.thread.QueuedThreadPool.new
    thread_pool.min_threads  = 5  # adjust as needed
    thread_pool.max_threads  = 50
    server.set_thread_pool(thread_pool)
    context = Context.new(nil, "/", Context::NO_SESSIONS)
    context.add_filter("org.jruby.rack.RackFilter", "/*", Handler::DEFAULT)
    context.set_resource_base(Environment.resolve)
    context.add_event_listener(MerbServletContextListener.new)
    context.set_init_params(java.util.HashMap.new('merb.root'=>; Environment.resolve,
        'merb.environment' => 'production',
        'public.root' => Environment.resolve('public'),
        'gem.path' => Environment.resolve('gems'),
        'org.mortbay.jetty.servlet.Default.relativeResourceBase' => '/public',
        'jruby.max.runtimes' => '1'))
    context.add_servlet(ServletHolder.new(DefaultServlet.new), "/")
    server.set_handler(context)
    server.start
    

Tweaking

At first blush our performance seemed to be pretty lacking. This required two tweaks: putting Merb in "production" mode and dealing with poor I/O due to logging. In the previous snippet you will notice that we set the merb.environment to production. Yes we lose the quick dev turnaround, but since there is a lot of Java in this project we usually have to recompile anyway which requires a restart anyway (phooey).

As for the I/O issue, a little digging revealed that shutting up Merb as much as possible would help reduce the amount of JRuby-level IO. In our config/init.rb we configure logging like so:

Merb::Config.use { |c|
      c[:environment]         = 'production',
      c[:framework]           = {},
      c[:log_level]           = :warn,
      c[:log_file]            = Merb.root / "logs" / "merb.log",
      c[:use_mutex]           = false,
      c[:session_store]       = 'cookie',
      c[:session_id_key]      = '_facet-store_session_id',
      c[:session_secret_key]  = '49411912879b879e13f89a9280c0f6aaa2e3ab58',
      c[:exception_details]   = true,
      c[:reload_classes]      = false,
      c[:reload_templates]    = false
    }
    

Here we set the environment to "production" again (yes, you need to do both). Also we upped the log level to "warn" which significantly reduced the amount of logging merb does. With these tweaks in place we found that the Merb port of our service was operating within about 80% of the level of performance we were getting from our pure-Java solution.

Benchmarking was done by running httperf tests against the resources we expose and comparing both the number of requests per second and the average response time. Given that the options for generating XML, HTML and JSON were all so much easier than what we were doing in the servlet version, we were willing to live with the performance hit.

That seemed like a great plan until I realized just how tangled some of the patches were. It isn't difficult for this to happen. As you extend the functionality of a system, you often refine earlier work. This was especially true in my case since we were introducing JRuby to a previously Java-only project and so there was a lot of experimentation and refinement. What I was really trying to do was re-write my commit history to separate the changes. Small cleanup commits could be collapsed with others to make the entire commit-set something that my teammates could easily understand.

Getting Started

The most basic kind of re-writing you can do is simply amending your last commit. On the command line this is accomplished with git commit --amend. The bigger rewrite-hammer is git rebase -i . This command will pop off all of your commits to the point specified by , provide you with a control file to edit and then re-apply your patches as directed.

The control file (a term I just made up) looks kinda like this:

pick 1cea777 Initial introduction of Merb.
    pick 5fe7a19 Favor XML over HTML as the "default" content-type.
    pick 6791134 Cleaned up the 'views' directory, out with the old, in with the new.
    pick ae6adac Put lots of back-navigation links to make it nice 'n' easy to use.
    pick 1da31ca Removed last vestiges of the RestServlet-related configuration and code.
    pick b9fe3b9 Added error views, updated error-handling and improvised content dispatch.
    pick bf92292 Routing cleanup. This is much more pleasant to read.
    pick ec74c63 An attempt to get RSpec working with Maven and JRuby.
    
    # Rebase 92ddc87..ec74c63 onto 92ddc87
    #
    # Commands:
    #  p, pick = use commit
    #  e, edit = use commit, but stop for amending
    #  s, squash = use commit, but meld into previous commit
    #
    # If you remove a line here THAT COMMIT WILL BE LOST.
    # However, if you remove everything, the rebase will be aborted.
    

The top section lists all of your patches, one per line. You can edit this section to do any of the following: * Reorder the commits (just move lines up or down) * Edit a commit (replace "pick" with "edit" or simply "e") * Remove a commit (just remove the line) * Merge with a previous commit (replace "pick" with "squash" or "s")

Think of this list like a program of execution. Once you save the file and return control to git, it will attempt to execute this program. I say attempt here because sometimes git runs into conflicts when it comes to re-ordering patches.

Editing Commits

When you mark a commit for editing (with "edit" or "e") git will attempt to apply all patches up to, and including that commit, and stop. This threw me at first because for some reason I had the unreasonable expectation that somehow the changes in the last commit would only be applied to the working tree and, perhaps, the index. Instead, that commit is fully applied, but all commits past that are pending. To add to my confusion, there isn't an easily accessible marker to indicate that you are in the middle of rebase (unless you frequently scan the .git directory as a matter of habit). At this point you could amend the commit (with git commit --amend) or insert other commits.

Sometimes I use this just to fix up the commit message. When I'm working on a new feature that has a lot of trial-and-error I tend to mark the first commit message with "WIP" to remind myself to review that patch and the ones following to see if I can clean them up. This is an area where git really shines. In a system like Subversion your audience (other coders who look at your changes) end up walking through whatever little mini-epic you've composed as you try things out, add things and remove things. This makes for some difficult reading for consumers of those patches and so a lot of folks tend to stack up big changes and then send in the big über-commit.

The problem with building the mega-patch is that you don't have a lot of scaffolding under you while you are building it. If you go off and explore something that doesn't turn out right, you generally have to do a lot of manual reverting. This simply sucks and is a waste of time. With git I can work in lots of incremental commits, then go back and edit them into something sensible once I'm ready to push changes.

Once you've finished doing whatever edit you want to do for that commit, simply type git rebase --continue and the remainder of the "script" will execute. If any other "edits" are in the pipeline, the process will repeat itself until the rebase is completed.

Squashing Commits

I looooove squashing commits in git. For any moderately complicated work, it's rare that I get it right the first time. Usually as I go along I find some mistake I made or find a refinement that cleans up the original work. As often as not, it's usually a couple of commits away so amending the last commit isn't going to help me. But hey, no worries! I simply commit the change and leave a log message for myself to merge it with another commit. Then, when I run the interactive rebase, I can simply move this commit up the list and change it from "pick" to "squash" (or "s") and let git merge the two commits.

When rebasing encounters a "squash" it will apply the changes in both commits. If successful, git will prompt you with a commit message file that includes the original commit messages of both commits. You can choose either, both or neither of these and save the file to continue.

Resolving Conflicts

Sometimes when applying a commit, git will run into conflicts and bail out on a merge. Whenever this happens to me I always have a little moment of panic as if I've broken something, but fear not—you can fix this. When this happens git tries to stage as many of the changes as it can. Any conflicts are left unstaged and need to be edited (look for the standard conflict markers), and then re-staged into the index. When you commit, git should show you the original commit message of the patch it was attempting to apply, which you can edit or keep. Once the commit succeeds, rebasing should automatically continue.

The Big Red Button

Sometimes you may just give up on a rebase. Either you can't commit the time to it, you don't really want to go through with it, or your patience has reached its limit. At any point, before the rebase has completed, you can execute git rebase --abort and your working tree, index and commit-log will all be returned to the state prior to starting the rebase. Think of this as the big red "stop" button for rebasing.

Playing Fast and Loose with Branches

If you come from other SCM systems like Subversion, you tend to treat branches as expensive and something you only use on occasion. With git, branches are cheap and easy like drinks in Tijuana. Be fearless! Not sure about some exploration? Make a branch!

If rebasing makes you nervous, and you're not entirely confident that aborting will save you, I suggest creating a branch on which to rebase. Simply create a new branch from where you're at with git checkout -b . You'll be immediately switched to that branch with your commit log looking exactly like the branch you came from. Here you can rebase, edit, remove, add and generally go crazy with your commits.

Once you've got your commits where you want them it's simply a matter of getting them back into whatever you consider your "main" branch to be and pushing any changes upstream. This works really well when you have rendez-vous point like a Github repo or an SVN server (we do a lot of git on top of Subversion at work). Assuming that you've been adding changes on your master branch, you might create a new branch off of master that you might call exp. On the exp branch you might rebase and wreak all sorts of havoc on the commit log. Once you have your commits where you want them, switch back to your master branch and reset it to point to the head of your upstream repo. If you're using git all the way, simply merge your exp branch over. If you're running git on top of Subversion, simply cherry-pick the commits on the exp branch (in order!!!) and push the changes upstream.

It's hard to stress how important it is to shift your mindset into thinking in terms of sharing patches. Treat your commits like individual, digestible changes. Even if you're working by yourself or in a small team, I still think there's value in being disciplined with your commits.

Instead, this is just a humble little notice about a humble little gem I put together today. It's called daemon-spawn and despite its simply terrifying name, it's really here to help all mankind. You see, I've been working like mad to stuff Merb smack-dab in the middle of an embedded Jetty project I've been working on. One of the last things I needed was a decent daemon-launcher/management gem-thingie to make it happen.

I cast about for an existing solution and found each a little lacking. The daemons gem had the executable name hard-wired to the output log name and didn't give me a clean way to specify additional arguments to JRuby (unless I wrote another wrapper script, to which I say "boo, hiss"). Then I looked at simple-daemon which seemed really promising. It was really really close to what I wanted but didn't extend very well as it required more and more class-methods. Yuck. I looked at daemon_generator, but it was very Rails-y and wanted to generate a bunch of code for me, which I didn't need. So I did what any honest, hard-working Ruby-dork does, and made my own!

It's simple—dead simple. Wanna see how simple? Here's a real-live echo server with daemon support:

#!/usr/bin/env ruby
    
    require 'daemon-spawn'
    require 'socket'
    
    class EchoServer << DaemonSpawn::Base
    
      attr_accessor :server_socket
    
      def start(args)
        port = args.empty? ? 0 : args.first.to_i
        self.server_socket = TCPServer.new('127.0.0.1', port)
        port = self.server_socket.addr[1]
        puts "EchoServer started on port #{port}"
        loop do
          begin
            client = self.server_socket.accept
            while str = client.gets
              client.write(str)
            end
          rescue Errno::ECONNRESET => e
            STDERR.puts "Client reset connection"
          end
        end
      end
    
      def stop
        puts "Stopping EchoServer..."
        self.server_socket.close if self.server_socket
      end
    end
    
    EchoServer.spawn!(:working_dir => File.join(File.dirname(__FILE__), '..'),
                      :log_file => '/tmp/echo_server.log',
                      :pid_file => '/tmp/echo_server.pid',
                      :sync_log => true,
                      :singleton => true)
    

But what if you have non-Ruby code you want to daemonize? Well my friends, that's what Kernel#exec is for and it works like a champ. See the README for the full details. And of course to view the README, you have to install the gem which means you have daemon-spawn in the bowels of your machine! Mwaaa haa haa haa! Oops…I've said too much…

In all seriousness though, I would like to thank the powers-that-be at work who were very gracious to let me open-source this. You should start seeing more of this kind of stuff from Evri soon. As always, your feedback, comments, critiques and patches are welcome.

We have a RubyGem written to handle registration and renewal that hides the HTTP and XML message bodies away from the user. You simply create a client, setup your initial registration and tell it to keep you registered.

require "rubygems"
    require "radar_love"
    client = Radar::Client.new("http://radar-dev")
    service = client.create('foobar', 'http://foobar:1234')
    service.keep_registered # fires up background thread
    

That last line is implemented with a Ruby thread that loops indefinitely, sleeping and then renewing the registration lease. But a funny thing happened while implementing this. When we just fired up irb and tried to run this part (without doing any other work), the re-registration thread never executed. Man, I had heard that MRI threads were "broken", but this is completely non-functioning!

Then I remembered a very handy page I ran across once about MRI Threading. This page is worth spending a little time with, but essentially because MRI threads are so-called "green threads" they aren't really giving you true concurrent processing. Instead they are merely a context-switching mechanism, and the circumstances under which those switches can happen are described in that spec page.

In the case of our little irb session, the re-registration thread only began executing when we did something in the main thread. We weren't executing anything in the main thread that triggered one of these context-switches (remember, we're merely sitting at an irb prompt waiting for the next line). So getting the runtime to execute a context-switch merely required us to do something in the main thread:

loop do
      puts "Howdy!"  
      sleep 5
    end
    

You may shake your head and mutter something derisive about this "hack". However, in reality, requiring your main thread to do something isn't terribly burdensome. If you didn't have any work to do in your main thread, you'd have to ask yourself why you created a separate thread in the first place!

You may also think that since MRI threads don't provide true concurrency, they're worthless. One major limitation of green threads in MRI is that no matter how many you start, they will only execute on one processor. If you have a large multi-core machine, MRI threads will not be able to take advantage of them.

However, that doesn't mean threads don't have their place in MRI environments. In the first example I mentioned (tailing logs and publishing summaries) we use a separate thread for the publishing activity. We could have done this entire action in a single loop, but the major downside is that we would essentially be relying on new lines in our log to appear to "crank" the mechanism forward. If we go a long time before we see another log line, our summary task will fail to execute.

IO.popen("tail -F /var/log/some.log").each do |line|
      update_statistics(line)
      if Time.now >= next_report_time
        # we might not get here for a while unless the
        # log lines keep coming
        report_statistics
      end
    end
    

It would certainly be possible to read from the file with a timeout that is based on how much time is left before the next reporting period. However at that point the code starts to get a little cluttered, so we go with the threaded approach only to take advantage of its context-switching properties. In our case, this is a perfect solution for what we're trying to accomplish.

If you come from a Java or .NET background, you may find that MRI threads fail to measure up to threading in those environments. It's absolutely true that MRI does not provide the same robust threading mechanisms that those languages do (JRuby, and perhaps IronRuby, being special cases). It doesn't mean that threads in MRI are worthless, you just need to understand them properly to know when to use them.

Meet Magit from Alex Vollmer on Vimeo.

It's not meant to be an exhaustive survey of magit (check the docs for all the details), but to show off some of the cool features this mode has. I found that spending just a little time with docs and learning this mode has already paid off in terms of increasing my productivity. Enjoy!

Changes:

1.0.1 / 2009-01-06

• Fixed a bug where generating help resulted in an infinite-loop

In moochbot, your main transaction screen has a standard tabbed-interface. Each tab is a different view of all your transactions. In a tabbed display you can only show one view at a time so it helps the user when you can provide some hints in the non-selected tabs. Anything that helps them figure out whether or not they want to click on something without actually having to click on it is, in my opinion, a great help.

So I wanted to add a number in the tab to indicate how many items were there. The most naïve way to implement this would be to issue three different SQL statements for the counts. However somewhere, in the back of my mind, I remembered that ActiveRecord has a feature known as the counter cache. The basic idea is to hook some additional code into the lifecycle of ActiveRecord's associations to update a single column in the parent record as you add and remove child records.

Like an iceberg, the bulk of this feature lay deep below the surface. The actual view-layer changes were minimal, but I had to jump through some hoops to get the counter-cache working correctly.

In moochbot, a User model object has multiple Transaction records. Each Transaction points at two separate User records: one for the lender and one for the borrower. All of a user's transactions are stored in the TRANSACTIONS table, each record differentiated by the STATE column.

In ActiveRecord-land we express these relationships in the model with three has_many relations for the User: one for items they are lending, one for items they are borrowing and all closed transactions. The first two are fairly straight-ahead:

class User < ActiveRecord::Base
    
      has_many(:lent_items,
               :class_name => "Transaction",
               :foreign_key => "lender_id",
               :conditions => ["state IN (?)",
                               %w(started lent returned overdue disputed)])
    
      has_many(:borrowed_items,
               :class_name => "Transaction",
               :foreign_key => "borrower_id",
               :conditions => ["state IN (?)",
                               %w(started lent returned overdue disputed)])
    
      has_many(:completed_items,
               :class_name => "Transaction",
               :finder_sql => 'SELECT * ' +
               'FROM transactions ' +
               'WHERE state IN (\'aborted\', \'finished\') AND ' +
               '(borrower_id = #{id} OR lender_id = #{id})')
    end
    

However the third relationship requires some custom SQL because we want all records that are in either the "finished" or "aborted" state and where the user is either the lender or the borrower. I looked into doing this with a simple :conditions option on the has_many relationship, but couldn't figure out how to specify the ID of the user.

One really important thing to recognize here is that the SQL is quoted with single-quotes. If the SQL is specified in double-quotes, the interpolation is evaluated too early and the id value is not the user record. Putting it in single-quotes defers evaluation until the proper time. I wish this were documented a little better because I was completely stuck until I stumbled across this post on the RailsBlaster blog. I had to enable debug-level logging for ActiveRecord to see that I was getting skooky IDs in my final SQL string.

To add a counter cache to the User record, you declare a :counter_cache option on the reciprocal belongs_to relationship. This seemed counter-intuitive to me since if I didn't already have one in place, I'd have to add one. It seemed more obvious to me to put it in the has_many relationship but that ain't the way ActiveRecord rolls. So the next step was to update the belongs_to mappings in the Transaction class:

class Transaction < ActiveRecord::Base
    
      belongs_to(:lender,
                 :class_name => "User",
                 :foreign_key => "lender_id",
                 :counter_cache => :lent_items_count)
    
      belongs_to(:borrower,
                 :class_name => "User",
                 :foreign_key => "borrower_id",
                 :counter_cache => :borrowed_items_count)
    end
    

The final step was to create a migration that would add the counter cache columns to the USERS table. Note that not only do we add the columns, but we also update everyone's counters.

class AddCounterCacheToUsers < ActiveRecord::Migration
      COLUMNS = [:lent_items_count,
                 :borrowed_items_count,
                 :completed_items_count]
    
      def self.up
        COLUMNS.each do |c|
          add_column :users, c, :integer, :default => 0
        end
    
        User.reset_column_information
    
        User.find(:all).each do |user|
          User.update_counters(user.id,
                               :lent_items_count => user.lent_items.length,
                               :borrowed_items_count => user.borrowed_items.length,
                               :completed_items_count => user.completed_items.length)
        end
      end
    
      def self.down
        COLUMNS.each do |c|
          remove_column :users, c
        end
      end
    end
    

So there it is. Hopefully that helps the next poor sod that runs into that same problem.

I've been a football fan since I was about five years old. I have studied this game for almost thirty years, so forgive me if I bore you all with a trite football analogy. This is the old saw known as "fast vs. quick". Players like Barry Sanders or Walter Payton were "quick", but not necessarily fast. In general, "fast" guys are wide receivers and corner-backs (the guys that cover the receivers). Receives generally have further to run than running backs, so they need what is called "breakaway speed" which is enough raw power to get away from the defender covering them.

Running backs however, need a combination of strength and the ability to move their bodies very quickly in a very small space. The difference between a gain of one or two yards and a breaking-off a twenty-yard run is usually a matter of inches. The runner has to be in just the right place at just the right moment and get just the right kind of blocking from his teammates. Quickness is the ability to shift weight, turn the body and plant a foot just a fraction of second faster than the next guy. It's what separates the good from the truly great.

Hall of Fame running backs rarely survive on pure speed alone. Heck, most backs that were "burners" in college are usually forced to play another position at the professional level. Those that try to rely on pure brawn to break tackles generally don't last long (see, Eddie George, Priest Holmes or any back that has played in Denver in the last decade). The really great ones have that ethereal quality known as "quickness".

So, I was thinking lately about "agile" in the software world and what it really means. The term agile, isn't clear enough—it could be fast or it could be quick. But which really matters? Which is the quality you really want to have?

My answer? You want the agility associated with quickness, not raw speed. Being "quick" in software might look like: * New requirements are handled in-stride and rarely, if ever, with The Big Re-Write * The time to produce a meaningful feature is measured in hours or days, not weeks * The time to produce a meaningful release is measured in days or weeks, not months * The system almost never looks like the original design

OK, so what would "fast" look like? Well… * Releases are buggy * Few if any tests * Code diarrhea * Quality is inconsistent * There are lots of dependencies * New requirements are an exercise in hammering reality into fixed model

Actually, none of those sound particularly "fast" to me.

When folks complain about the ineffectiveness of "Agile", I sometimes wonder if they aren't simply confusing fast with quick. There isn't any methodology or toolset I know of that will make you faster. Your raw speed is determined more or less by the skill of your team. Each member transfers a concept into code at a relatively consistent rate, so you simply aren't going to go any faster unless you change your team. However I think you can acquire new skills to improve your quickness.

He came from the .net side of things and had been doing contracting for quite awhile. Recently though, his shop closed up and he was between gigs. He hadn't been exposed to Ruby and thought it sounded kind of interesting. He was, as he put it, "a Windows-guy", but basically apologized for it saying that "everyone around here" runs Windows. Really? You mean like the four-out-of-five-laptops that are Macs in this café??!!! (Note to self: this fella doesn't seem too observant.)

So while I was trying to politely end the conversation to get back to my hacking and an IRC-meeting, he rambled on about his personal work philosophy. God knows how long he droned on, but his M.O. essentially boiled down to "Hey I don't care what I work on. Languages are all the same, why fuss over this one or that one?"

It stopped me in my tracks.

Are you kidding me?

Look, there is something laudable (in a kind of John-and-Yoko-stay-in-bed-for-peace way) about The Big World view where all the geeks get along and sing side by side at the great binary campfire. At a certain level arguing over this language vs. that language or this tool or that operating system is frivolous waste of time. Much of our "values" in these things are based merely on opinions, aesthetics and feelings. It's like mailing list arguments over whether or not the Enterprise could defeat the Galactica. Ladies and Gentleman on your left is Dork City…

But what this really told me more than anything else was that this guy was simply indifferent. You don't have to agree with my way of doing things, but when we talk about them I at least expect you to have an opinion and to be able to back that up with some reasoning. This guy's indifference stemmed from the fact that he treated this like a job.

"Passion" is a word that is overused in this biz. Every time I'm in a debrief about some candidate and the notion of "passion" comes up I started checking the room for forks to shove in my eyes. I don't care about passion for The Company or The Idea. That's rah-rah B.S. that I have no stomach or patience for. What I do want to see in a co-worker is someone who cares about what they work on and how they do it. This is your day-to-day existence. This is what you spend a lot of waking hours doing, being away from your loved ones. You damn-well better care about how you want to spend that time.

If you wanted to fail spectacularly in a job interview with me, one way to score a boat-load of demerit-points quickly is give an indifferent answer to the question, "What do you like to work on?" When people shrug their shoulders and say, "Oh, I'll work on anything", it tells me that they don't care. The pay is the same, so why fuss over the details?

Now I understand that some interviewees won't give me a straight answer because they expect that I want the answer an automaton would give. There are certainly plenty of shops that just want "human capital" in much the same way generals in WWI wanted troops (see fodder, cannon). But even so, I'm not sure that's the kind of person I want on my team. Show a little back-bone for pete's sake! Do you do this job just because it pays the bills? Are you so beaten-down by the industry and your experiences that you have no hope of it ever getting better? Are you just trudging forward in your profession because you don't know what else to do?

I don't care so much whether or not we agree on tools, languages or operating systems, but I hope to God you care about how you spend your time on this earth in this profession. Don't sleepwalk through it.

When Columbus reached the New World, he burned his ships. As a result his men were well-motivated.

This quote comes from the character of Marko Ramius, as played by Sean Connery, from one of my all-time favorite films, "The Hunt for Red October". I love that quote in particular because I always think of it whenever I decide to tweak my process, my work environment or my tools. For all but the most trivial changes, you usually have to immerse yourself in it. So I usually "burn my ships" by leaving behind the old way of doing things to really burn the new way into my brain.

Sometimes the new way turns out to not be so great. That's fine because that's a satisfactory result of the experiment. If the "new way" turns out to suck, there's just as much to learn (if not more) from a real crash-and-burn than success.

The first big "ship-burning" I did was very early in my career. I had been a few months into my professional developer career doing Java and decided that I really liked this Unix thing. So I completely wiped out my Windows NT work box and put one of the first versions of Red Hat on there. I figured if I was going to get good at Linux I was going to have to just jump in the deep end of the pool and learn to swim.

It turned out to be an absolutely wonderful decision. Throughout the years I moved through several distributions: Red Hat to Mandrake to Gentoo to Ubuntu. Each new distribution taught me something more about Linux than I knew before (especially true for Gentoo).

I've done the same trick with editors too. When I first started out I used vi for my Java development. I tried using the first IDEs (JBuilder, etc.) but they were all terrible. Plus, at that time a serious Java IDE for Linux was still a twinkle in Erich Gamma's eye. I had a co-worker who was an XEmacs power-user and it looked like he got a lot done so I figured I'd give it a go—especially since I had on-site technical support. This was my first experience with Emacs and I never really got beyond basic editing. I wrote a couple of macros but that was about it.

Then NetBeans rolled around and I was more than happy to start using it. I had a decent intellisense feature and built-in Ant integration, both of which were a significant step forward to me. I ran with that for a couple of years until I started a new job where everyone was using this new-fangled thing called Eclipse. I tried to hold-out with NetBeans for a while, but after watching some others pull off some seriously cool tricks in Eclipse I knew it was time to give it a go. I've ridden that Eclipse horse for damn-near six years and I'd be willing to put my knowledge of Eclipse-editing arcana up against anyone.

Several years laster, on the operating system front, another big change was coming. I had been a die-hard Linux fan since my switch back in the Old Days, but by winter of 2005 I had run into a tangled knot of Linux-related headaches that really sapped the fun out of it for me. Between home servers and 64-bit laptop (in 2005!) I spent more time just trying to keep my machines up to date and running than actually getting anything productive done. However, it wasn't until I went to my first Seattle Ruby Brigade meeting (when everyone there had a Mac) that I seriously considered switching operating systems.

This was the same time that I discovered Ruby. What attracted me to the language was not only the beauty of the syntax, but the underlying philosophies of pragmatism, simplicity and no-bullshit attitude. There seemed to be a similar attitude in Mac users and I found this approach appealing. This is not a knock on Linux. I love Linux, I just don't want to use it for my desktop OS anymore. I don't have the patience to fiddle with things that much anymore. I have lots of ideas and I'll never get anywhere if I get bogged-down with irrelevant minutiae and yak-shaving.

When I switched to the Mac I got myself a license for TextMate which has definitely been the editor-of-choice for lots of Rubyists. However, as I blogged about before, it's now time to get off the TM-train and switch (back) to Emacs.

So what's the point of all of this? Am I just thrashing? Am I desperately throwing myself at each and every new tool on the block? You might think so, but I actually have some discriminating taste. I do try out a lot of tools because I'm always looking for some kind of edge, anything that I can leverage to get stupid busy-work out of the way. As I get older I have less and less patience for this sort of thing for productivity taxes. I just want to get something important done.

So, from time to time, a little alarm goes off inside my brain to remind me that it's time for a change. Sometimes it's something as simple as changing my terminal or editing font. Other times it's things like learning a new language. Regardless, all of these things keep me from getting mentally lazy. Once I get mentally complacent, it's game over. So consider if perhaps it's time to "burn some ships" just keep things interesting.

Good heavens, some day I may even do something as crazy as switch to a Dvorak keyboard layout. That just might make my head explore—in a good way.

Yesterday the city of Seattle was paralyzed by the fear of snow (which arrived today). As a result downtown was a ghost-town. That meant that we could get sandwiches from Salumi (one of Seattle's culinary gems) in less than the usual hour-long wait.

Even on a "quiet" day, the process still took about thirty minutes to order, receive and pay for our sandwiches. As we walked back to the office we started chatting about ways they could be faster. They could make sure that people are paying as quickly as they are getting their sandwiches made. They could make sure that the full menu is displayed further back in the line so customers know what they want as they arrive.

We even ranked some of the local sandwich shops in terms of speed. If you want an incredible sandwich and have the time, Salumi is hard to beat. Next up would be our beloved Tat's. Finally, you can go to a chain place, like Jimmy John's, and get a decent corporate sandwich in unbelievable time.

So then I asked myself the question, "why doesn't Salmui try to go any faster?" Wouldn't their customers be happier? The lines are ridiculous at times and you can blow a whole lunch-hour just on getting a sandwich. The place is tiny too. It's near-impossible to eat it there because of the wait for a table. Wouldn't they like a bigger location to service more customers?

And then it hit me: they don't go any faster and they don't grow any bigger because it wouldn't make them any happier to do so. The folks behind the counter clearly love what they're doing. It's not that they're indifferent to their customers, it's that they seem to be doing quite well the way things are and don't feel a need to "grow". I think there's something very laudable in that attitude.

Anyway, one day it occurred to me that it would be funny (to me at least) to have a very simple, single-serving site that told you whether a certain song or artist was, in fact, Freedom Rock. Note the capital letters. I'm not talking Toby Keith "freedom rock" with waving flags in the background and lots of shiny pickup trucks. I'm talking about the rather odd, rag-tag collection of songs that made it to this album. Try playing with the site a bit, then check out the Urban Dictionary's entry on Freedom Rock.

This leads to one of the pillars of my personal philosophy:

Amuse yourself first, worry about your audience later.

I wrote this primarily because it make me laugh…a lot. I don't know why. It's dumb, it's juvenile and it's totally worth the $10 I paid to register the domain-name.

But stupid-hippy jokes don't make up the whole picture. The other half of this fun little excursion was figuring out how to do this with as little code as possible. At first I wrote this as a flat Merb application. Unfortunately I wanted to host this on Dreamhost and they are still on Ancient Ruby (read, 1.8.5) and the newest Merb wants 1.8.6. That was a deal-breaker. So then I took a look at Sinatra, and just like the bobby-soxers of the 40's I fell in love. Sigh…

If you haven't had a chance to look at Sinatra, you should do yourself a favor and check it out. They call it a DSL for the web, not a framework. It has the nice terse syntax of Camping, but has source that gives me half a chance to comprehend. It's short and sweet and gives enough things that will make sense to Rails-veteran, but with little extra cruft. The source is up on GitHub (with a shocking number of commits for such a simple project).

• Receiving email works much much better when you configure your MX records correctly
• Mixing ERb and HAML is a really really bad idea
• I'm probably using RubyGems in my Rails app in about the stupidest way possible
• Good dynamic form validation with JavaScript is actually quite tricky
• Squashing commits in Git is still pretty damn cool.

That's all for now. Good night and good luck.

The secret side-project that I've been referring to in various posts finally reached its Version One milestone tonight. I closed out the remaining V1 issues off of my to-do list and suddenly realized that I could turn it loose on the world. So, without further ado, I present to you: moochbot. Check it out and see what I've been up to all this time.

I'd love to hear any feedback folks have; what you love and what you hate. I have a ton of V2 features planned so moochbot will certainly continue to evolve. It's running on a single Slicehost slice so it's not real high-powered. Your patience is appreciated.

I'll have plenty of blog-material from this experience to write up later, but for now I'm just going to enjoy the satisfying feeling of getting something done. Enjoy!

For some reason I've always been a little bothered by this and have always felt like it should be avoided. I dislike this way of speaking about a system because of how easy it is to personalize code that someone has written. I think it's simply too easy to make a statement that sounds like a criticism of the person instead of the work. A criticism of the latter can be hard enough to take, but when it's misconstrued with a personal critique it's very easy for feelings to get hurt and team morale to take a nose-dive. In my experience it takes far longer to gain the trust of your teammates than it does to lose it.

The other reason I don't like it is that it reinforces ownership boundaries around particular parts of the code. In an ideal world your team members would be able to move fluidly about the codebase depending on the features to be implemented. In reality this can become difficult, but like a lot of thinks in life I think it's a good "stretch-goal". So when we speak in personal pronouns instead of component names it seems to reinforce these barriers (or maybe it simply highlights them.)

I've found that it's difficult to avoid talking about systems in these terms. I have to make a conscious effort to not do this in much the same way I have to remember to floss. Both seem like unnatural activities that require a lot of discipline to maintain. I can understand why I might be inclined to forget flossing regularly, but it's not as clear to me why all of us developers want to personify code in our daily speech. Any ideas?

Like a good boy I started out with the simplest thing that could work which was to simply call mailers in my model. However, I wanted to limit the number of activities performed during a request to keep the app feeling responsive. So I decided that asynchronous mail delivery was a "pre-launch" feature that I had to have.

I looked at a variety of background processing tools, including Bj, Starling/Workling, Spawn and AP4R. Each had its strengths and weaknesses but none of them felt like the right fit. My research criteria included: * Job persistence via the database * Something that could get a Rails environment cheaply * Runs outside of the Rails processes * Minimum fuss to get it running

In the end the one that hit the sweet-spot best was delayed_job. It had the DB persistence I was looking for, but didn't source the Rails environment for each worker and it was extremely simple to plumb it into my app.

Refactoring

The first step was creating the DelayedJob worker classes; one for each mail action. At first this turned into a big pile of five-line classes, so to keep things organized I put these all in app/models/jobs and put each class in the Jobs module namespace. This was better, but not good enough so the final step was putting all of the worker classes in a single file, app/models/jobs.rb.

The second step was to find every place in the model where I called my mailer classes directly and replace them with calls to enqueue the appropriate worker job.

Here is what things looked like at first:

class UserObserver < ActiveRecord::Observer
      def after_create(user)
        unless user.current_state == :latent or user.is_a?(Admin)
          UserNotifier.deliver_signup_notification(user)
        end
      end
    
      def after_save(user)
        if user.current_state == :promoted
          UserNotifier.deliver_signup(user)
        else
          UserNotifier.deliver_activation(user) if user.recently_activated?
        end
      end
    end
    

Then the UserObserver was refactored like this:

class UserObserver < ActiveRecord::Observer
      def after_create(user)
        unless user.current_state == :latent or user.is_a?(Admin)
          Delayed::Job.enqueue(Jobs::UserNotifierSignupNotificationJob.new(user.id))
        end
      end
    
      def after_save(user)
        if user.current_state == :promoted
          Delayed::Job.enqueue(Jobs::UserNotifierSignupNotificationJob.new(user.id))
        else
          Delayed::Job.enqueue(Jobs::UserNotifierActivationJob.new(user.id)) if user.recently_activated?
        end
      end
    end
    

With the following workers (abridged):

module Jobs
      class UserNotifierDisconnectJob < Struct.new(:user_id)
        def perform
          UserNotifier.deliver_disconnect(user_id)
        end
      end
    
      class UserNotifierResetPasswordJob < Struct.new(:user)
        def perform
          UserNotifier.deliver_reset_password(user)
        end
      end
    
      class UserNotifierSignupNotificationJob < Struct.new(:user)
        def perform
          UserNotifier.deliver_signup_notification(user)
        end
      end
    
      class UserNotifierStartDisconnectJob < Struct.new(:user_id)
        def perform
          UserNotifier.deliver_start_disconnect(user_id)
        end
      end
    end
    

Testing

Prior to switching to asynchronous processing, mail delivery was triggered within my models, either via Observers or as Procs attached to state transitions (I'm using the acts_as_state_machine plugin). Therefore my tests had loads of assertions that various state changes in the model resulted in direct email delivery. In the asynchronous model of course, that changes slightly. While the state change ultimately ends in mail delivery, it only happens indirectly.

So here I had a big pile of tests that asserted that poking the model in certain ways resulted in a mail delivery. In my unit-tests I really just wanted to test the interaction between the models and DelayedJob. After all, if something went hay-wire during mail delivery the culprit would likely be my new worker classes, not my model.

However, for my integration tests I still wanted to keep the assertions about actual mail delivery since that was an important part of the stories. I could easily do this by monkey-patching the DelayedJob::enqueue method to call the worker's perform method directly. In my unit-tests I monkey-patched the DelayedJob::enqueue method to work more like a mock object which added some inquiry methods to check that it had been invoked correctly.

In isolation this worked great, but running all the tests together resulted in a number of random failures. I've run into this enough times to recognize that some tests were somehow poisoning the run-time environment for the others. It turns out that my two approaches were incompatible with each other unless I was very diligent about cleaning everything up properly. I will admit with red-faced shame that I punted. I did the lamest thing one could possibly do and redefined DelayedJob::enqueue for all of my tests and kept all of my original assertions. I'm not proud of it, but it does work.

Running in Production

The next trick was getting this all running in a production environment and I needed to figure out how one or more workers would be started and kept running. While it's great to have this decoupled from the Rails environment, it means that it's a separate process that needs to be managed.

My solution was to use the daemons gem to create a couple of scripts. Then I used Tom Preston-Warner's god, to monitor my process. The scripts look like this:

#!/usr/bin/env ruby
    
    unless ARGV.size == 1
      $stderr.puts "USAGE: #{0} [environment]"
      exit 1
    end
    
    RAILS_ENV = ARGV.first
    require File.dirname(__FILE__) + '/../config/environment'
    
    Delayed::Worker.new.start
    

The "control" script looks like this:

#!/usr/bin/env ruby
    
    require "rubygems"
    require "daemons"
    
    def running?(pid)
      # Check if process is in existence
      # The simplest way to do this is to send signal '0'
      # (which is a single system call) that doesn't actually
      # send a signal
      begin
        Process.kill(0, pid)
        return true
      rescue Errno::ESRCH
        return false
      rescue ::Exception
        # for example on EPERM (process exists but does not belong to us)
        return true
      end
    end
    
    if ARGV.size == 1 and ARGV.first == "status"
      pidfile = "/var/run/delayed_job_worker.pid"
      if File.exists?(pidfile)
        pid = open(pidfile).readlines.first.strip.to_i
        if running?(pid)
          puts "delayed_job_worker is running (#{pid})"
        else
          puts "delayed_job_worker is NOT running (#{pid})"
        end
      else
        puts "delayed_job_worker is NOT running (none)"
      end
    else
      Daemons.run(File.dirname(__FILE__) + '/delayed_job_worker',
                  :backtrace => true,
                  :log_output => true,
                  :dir => File.dirname(__FILE__) + "/../tmp/pids",
                  :dir_mode => :normal,
                  :multiple => false)
    end
    

The first script is merely the smallest amount of chrome required to start a worker. Note that I'm using John Barnette's version of delayed_job which gives us that nice little worker riff. The second script is essentially the daemons wrapper around my worker. For a little extra goodness I added my own "status" command which can be handy for debugging.

Getting those to work properly took a little bit of testing by hand. Fortunately, the entire solution is really a series of layers applied on top of each other. Once you convince yourself that an inner-layer is working correctly you can move on to build the outer layers.

The next step was to create a proper god configuration. I have more than one thing to monitor on my setup so I have a master god configuration that includes sub-configurations. My "main" configuration is a simple one-liner: God.load "/etc/god/*.god" My application-specific configuration looks like this (with a few edits for public consumption):

RAILS_ROOT = "/var/www/moochbot/current"
    
    God::Contacts::Email.message_settings = {
      # your config goes here
    }
    
    God::Contacts::Email.server_settings = {
      # your config goes here
    }
    
    God.contact(:email) do |c|
      # your config goes here
    end
    

After setting up some default notification details, we get into the meat of defining our "watch":

God.watch do |w|
      w.name = "delayed_job_worker"
      w.interval = 10.seconds
      w.start = "#{RAILS_ROOT}/script/delayed_job_worker_control start -- production"
      w.stop = "#{RAILS_ROOT}/script/delayed_job_worker_control stop"
      w.restart = "#{RAILS_ROOT}/script/delayed_job_worker_control restart"
      w.start_grace = 10.seconds
      w.restart_grace = 10.seconds
      w.pid_file = "#{RAILS_ROOT}/tmp/pids/delayed_job_worker.pid"
    
      w.uid = "deploy"
      w.gid = "root"
      w.behavior(:clean_pid_file)
    

Next we need to define our transitions. My first attempt at this failed because I was missing these and my watched process was stuck in the "unmonitored" state. It's worth spending some time reading the docs on the homepage since at first glance the configuration wasn't obvious to me.

  # determine the state on startup
      w.transition(:init, { true => :up, false => :start }) do |on|
        on.condition(:process_running) do |c|
          c.running = true
        end
      end
    
      # determine when process has finished starting
      w.transition([:start, :restart], :up) do |on|
        on.condition(:process_running) do |c|
          c.running = true
        end
    
        # failsafe
        on.condition(:tries) do |c|
          c.times = 5
          c.transition = :start
        end
      end
    
      # start if process is not running
      w.transition(:up, :start) do |on|
        on.condition(:process_exits)
      end
    

Finally I specify some resource-consumption boundaries to make sure that my little worker daemon doesn't take over my box:

  w.restart_if do |restart|
        restart.condition(:memory_usage) do |c|
          c.above = 100.megabytes
          c.times = [3, 5]
        end
    
        restart.condition(:cpu_usage) do |c|
          c.above = 50.percent
          c.times = 5
        end
      end
    
      w.lifecycle do |on|
        on.condition(:flapping) do |c|
          c.to_state = [:start, :restart]
          c.times = 5
          c.within = 5.minute
          c.transition = :unmonitored
          c.retry_in = 10.minutes
          c.retry_times = 5
          c.retry_within = 2.hours
        end
      end
    end
    

So without further ado, let's start with the New York Times:

If you remember my original post, I favored the tabular displays generally over map-based views. My favorite part of this is the electoral vote tally on the left-hand side. To me, the most important information is the electoral vote tally and the Times has done a good job of highlighting this both by making it a larger font and putting it on the left-hand side. Then as my eye moves to the right I can drill down to state-by-state detail. The color codes provide a nice way to get a "feel" for the state of things at a glance.

Next stop, CNN:

What you don't see here is the completely unnecessary animation of the left-hand counters like some alarm-clock from the seventies. The other three panels "flip" to various local races. Instead of using space show multiple data-points, CNN chose to use time. This feel like a real waste of space and my time. When compared with the Times site, there is really no contest.

Let's move on to MSNBC. These guys get bonus points for including the Senate races in aggregated form—a nice touch if you want to take the pulse of all the races. This is a nice tight display of a lot of information.

Again, I could do without the map. It's not that I don't like maps (I like them quite a bit), but they don't tell me the most important information—the electoral college votes. It's interesting to see the geographic proclivities but it's essential that I know the electoral college votes. The geographically large, but sparsely-populated states in Big Sky Country and the Midwest distort our initial perception of the data. Their size belies their actual value in the race and so we have to constantly look elsewhere for electoral college vote totals. Unfortunately, this graph doesn't tell that story. I can't look at Ohio in this chart and understand its value in the race.

Next, the Washington Post:

This is pretty uninspiring and uninformative. Oh look! Pictures! Just in case you forgot what the candidates looked like. However, if we start clicking the tabs, we get a little more meat:

I'm not sure why the map is all-gray, but the I like the bar at the top. This seems to be the new trend this year in info-graphics. I think for highlighting the changes in the balance-of-power, the line chart does a good job.

Now we move onto USA Today…

That's it folks. That's all you get on the front page, up in the right-hand corner. I think that pretty much sums up the estimation the editors of McPaper have of their audience. Let's, for a moment, try to treat this with a serious eye. Could somebody please explain what the blue boxes on the right-hand side are for? Do you click them? What do they do? Why are they the only ones shown? Can you see others? This is what Edward Tufte would call "chart junk".

On to the once-venerable Wall Street Journal…

Same old map, so few details. I can't tell if the lack of titles on the shapes indicates that the Journal's editors assume their audience can actually tell the states apart, or that they just couldn't be bothered. Like the Washington Post we get the candidate's faces. While it fits the Journal's famous stodgy etchings theme, it doesn't bring much information to the table. Oh yeah, just in case you lack the skill to compare numbers, the big checkbox next to Obama's name serves to tell you who won. Sheesh.

One interesting note is that this is the first infographic of the major sites I surveyed that included results for any of the other candidates. It's interesting to note that just the simple quantity of digits gives you some idea of the magnitude of the votes the two major-party candidates get.

Check back in four years and we'll do it again!

Let's say that the processing of writing your program is like launching spacecraft. Ideally you would like to get from launch to cruising around in space as quickly as possible. The Star Trek universe solves this quite elegantly with the transporter. We don't put you in a box and launch it, we break your atoms apart and transmit them to another location! That's pretty cool, but maybe we're just not quite that cool yet. Another solution is one proffered by the Star Wars universe. A ship like the Millenium Falcon can leave just about any planetary atmosphere any time it damn well pleases without the use of special equipment. It just flies away. Not bad.

However, here on earth, our primitive space craft need a tremendous amount of disposable apparatus to reach escape-velocity. The proportion of useful vehicle (the shuttle) to the orbit-busting mechanism (the rocket boosters and fuel tanks) is a staggering 5.4:1 (based on liftoff weight).

Command-line parsing in code exhibits a similar disproportion. The interesting part of your app isn't the command-line parsing. Why should it take up such a disproportionate amount of space in your code? Those boosters quickly become "space-junk", once the launch vehicle has left Earth; expensive trash that is never to be used again.

Space-junk is dangerous for the next guy that wants to launch into space. It's also dangerous for the folks on the ground as it may decide to come crashing back down to earth. And those boosters have also been responsible for one of the worst disasters in NASA's history. If we could only get rid of those damn things the whole space program just might be a little better off. Unfortunately physics, and our current space-flight capabilities currently require them.

But our code is a different story. We don't have such physical barriers that handicap us. Any barriers we run into are usually of our own making. So why not try to reduce those as much as possible? Why say in more code, what you could say in less? Wouldn't that cut down on bugs? Wouldn't that ease the burden of maintenance? Wouldn't that reduce the amount of information overhead you have to maintain each time you revisit that code?

Command-line parsing is a stupid, menial task that should require very little attention. By extension, it should only be given a stupid, menial amount of code to make it work. We have big ideas! They shouldn't get bogged down by handling command-line options!

This is why I wrote Clip. Clip is an expression of the need to make the simple things simple, but no simpler. If you have modest command-line parsing needs, Clip rewards you with minimal investment. If you need something trickier, Clip allows you to say a little more to it and gain more benefits from it. You get to decide how much you want to engage—not the library.

This is one of the things I like about Ruby. The language is extremely flexible which gives me a lot of ways to "pack" ideas into code in a variety of ways. Having more than one way to do things isn't all that useful by itself. But it's essential when you want to write expressive code. Things like object literals, or one-line control-structure alternatives help me keep the lines of code proportional to the ideas they express.

This is also something I find challenging to do in Java. In languages like Java, even just creating a collection of objects requires quite a bit of code:

import java.util.*;
    
    public class Designer {
      public void makeItWork(List<Trash> trash) {
        // today's challenge: convert trash to wearable garments
        List<Garment> garments = new ArrayList<Garment>(trash.size());
        for (Trash t : trash) {
          Garment g = new Garment(t.getName());
          garments.add(g);
        }
    
        submitToJudges(garments);
      }
    }
    

In Java we often solve this by pushing all of that code into a private method that is named something meaningful. This works pretty well, but does tend to result in an explosion of "helper methods". Sometimes folks take the "cheap" route and simply prefix these riffs with explanatory comments like "convert each Trash into a Garment". I'm not a real big fan of this. Generally I don't care about the object conversion in the first place because the rest of the code is presumably doing something interesting with the Garments and I don't give a damn about the Trash.

So let's look at it in Ruby:

class Designer
      def make_it_work(trash)
        submit_to_judges(trash.map { |t| Garment.new(t.name) })
      end
    end
    

By my count there are five lines in the Java example (including the comment) just to convert trash to garments. In contrast I boiled that down to one line in Ruby. OK I could have done this in two lines if you think that's too much of a long-train. But I think there are couple of important points here: * The importance of the concept being expressed diminishes from left to right * The attention-span of the reader diminishes from top to bottom

The Ruby example beats Java on both counts. I don't waste a lot of the reader's attention span up-front on book-keeping details (in the vertical space) and I state the important thing I'm trying do (submit my top Foos) quickly (on the left). The details of which Garments I'm dealing with are merely a qualification of what I'm trying to do.

How you handle these two dimensions is greatly affected by both the language you use and the APIs you deal with. This is one of the reasons that I do not find the use of scripting languages for Java's Swing API all that compelling. Scripting languages like JRuby or Jython help me with the horizontal space, but don't do a damn thing for the vertical requirements. With an awful API like Swing I have to say a lot of words to make it go, regardless of the language I do it with.

Getting back to my dumb example, being on such a small scale this may not seem like a large impact. But multiplied several times over to match the size of most projects, this kind savings can really pile up. The difference in the amount of code required by these two approaches is manifested in a savings in cognitive investment required to grok these projects. This is the very essence of maintainability and sustainability. Anytime you can do more with less, you come out ahead.

At great peril to my own geek cred, I will say that this is why I find The Lord of the Rings to be such an awful piece of writing. It is so full of peripheral and non-essential information that finding the real story or characters requires extraordinary patience and concentration on the part of the reader. If Tolkien had been more concerned with the story and less with "world-building" I'll bet he could have gotten that story boiled down to a single book.

Now I realize that a lot of folks love the Tolkien books for the very reasons I criticize it. That's fine, that is an argument about aesthetics, not facts. However I would strongly argue that "world-building" in your code is a bad idea. I think you're much more likely to build a decent piece of software if you pack your ideas tightly like a William Gibson novel than as a sprawling "trilogy" of epic code. Go ahead, prove me wrong. I double dog-dare ya.

OK, so by this point any credibility I may have had is gone. Look at the size of a post about saying more with less. In the hope that you might be lazy and like to skip to the end: Do as much as you can…with as little as possible.

Since switching to the Mac over a year and half ago, I've used TextMate as my non-Java editor. TextMate is great text editor. It's relatively simple to extend, has a very active community and only takes a little investment before a user gets productive. But I'll admit that in the last month or so my eyes started wandering from TextMate—I felt like maybe I was ready to go back to Emacs.

Perhaps it's worth a little history first. Back in the Dark Ages of Java Development (anyone remember Gnu Server Pages?), I was a pretty hard-core XEmacs guy for my Java dev. At the time Java IDEs were painful exercises in waiting and crash-recovery. JBuilder, early NetBeans, Visual Cafe were all valiant attempts but terrible failures. So I rode with XEmacs until the IDEs caught up.

Then one day I met Eclipse. Unlike any other Java editing tool, Eclipse was much more than a text-editor. Rather than treating your code like text to be manipulated, Eclipse actually understood your code. This allows you to think in terms of manipulating Java syntax and symbols instead of simple text tokens and lines. For a static language like Java this is incredibly powerful since you can determine up-front all of the possible things you might want to write. This is why a full-on "intelli-sense" feature for dynamic languages is hard to do as well in static, compiled languages.

I spent the next five years learning just about every nook and cranny of Eclipse and would put my raw Java-writing skills up against anyone. Even now, I still use Eclipse for Java and I can't imagine going back to a plain text-editor—even a souped-up hot-rod like Emacs. This says less about the deficiencies of any text editors as it does about the requirements for developing in Java. Writing Java without all of the code-completion and refactoring tools an IDE like Eclipse has would simply exceed my pain threshold. So while Eclipse has become quite bloated lately and, when coupled with tools like Maven, is an exercise machine-servitude, it will be a cold day in hell before I give it up for Java. (NB: I plan on giving up Java-dev first.)

Now I'm writing a lot more non-Java than I did a year ago. That's what brought me to TextMate. The integration for Ruby, HTML, CSS, JavaScript and shell was outstanding. I bought the TextMate book and was on my way to editor ass-kicking.

Unlike my Java days, now I identify myself less with a single language. Back in the day I was a Java Programmer™ dad-gummit. It was The One Language To Rule Them All and it served me well. Do you remember those days? Yeah, that was also the time when certified Oracle dudes were writing their own checks. But as time went on, I noticed that those Oracle guys pretty much knew how to do one thing and if and when the Day of Database Reckoning ever came, these guys were gonna be left standing in the cold wondering what the hell happened. I think that day has come and those dudes are becoming the equivalent of COBOL programmers—relegated to propping up legacy systems and talking wistfully about "the good ol' days".

So it got me to thinking about what I needed to do to survive in this business. Clearly I was going to have to evolve. So I started looking around for some inspiration and came across Ruby. Right now it's my favorite language, but it's not the only game in town. Ruby borrows a lot of features from other languages and a lot of other Ruby-ists are also spending time with other languages like Haskell, Erlang, io, and Lisp just to get exposure to other ideas. Moving away from a single-language focus to broader interests has been an important part of moving from rookie to journeyman. Evolve or die, right?

So how does this relate to Emacs? Well, like a lot of folks, I've been interested in learning more about functional programming. The problem is, I simply haven't had the time to crack open the Erlang book and bend my brain in new ways. Don't get me wrong, it's on the list, but it's going to be a while before I get to it. So I was eager to dip my toe in FP without a major commitment. Enter ELisp.

Before I dropped out of college, my brief career in academic computer science was spent with Scheme. My time in school didn't end well and so I've associated Scheme (and by extension, Lisp) with that particularly unpleasant time in my life. I've only recently overcome the shakes when I see car or cdr. But a little time and some professional success has healed old wounds. In my first go-round with Emacs, I was really just using it as a power-editor. The internals of Emacs pretty much escaped me and in my mind there was a pretty major barrier between an ELisp user and an ELisp writer.

This time I've made a concentrated effort to learn more about how Emacs works. I've been racing through the O'Reilly Emacs book and this time it makes sense. I don't know what the hang-up was before, but this time it all seems so logical! The entire editor is simply a collection of ELisp functions and variables built on top of other functions and variables. It's all self-documented and incredibly dynamic. Wanna try something out? Just put a little Lisp in your *scratch* buffer and evaluate that sucker. Hell, in Eclipse I'd have to download the freakin' source and compile it or swim in XML writing an extension. That's incredibly powerful—the barrier to "trying stuff out" is mind-bogglingly low.

So while I've definitely take a short-term hit by trading in TextMate for Emacs, but I'm reaching productivity-parity pretty quickly. A couple of modes/libraries I've found that I've really liked are:

• Steve Yegge's JS2 mode
• Rinari, for Rails
• Org-mode
• Magit for Git integration

I plan on spending some quality time with some of the various snippets libraries too to port over some of the templates that TextMate uses. For something like Rails development, which is about the most idiomatic programming I can imagine, having short-hand snippets is a major productivity boost.

So why go through this? After all, switching editors can be like converting from Catholicism to Judaism for some. Well, I think I'm doing this because:

• I've already overcome the initial steep-learning curve of basic editing in Emacs
• I get a nice, no-commitment introduction to functional programming
• I get a great environment to learn new stuff in
• With a little effort and a willingness to learn, I can make this sucker to anything
• Emacs has quite a track-record. It's close association with Unix is no accident.
• Emacs customization is easy to write and a snap to track in SCM

I can't say that I'll use Emacs for everything. I'm getting into Cocoa development too and it looks like doing that outside of XCode is a fool's errand. But that's okay. I don't need a single tool to do everything for me, just a handful of tools that help me get a variety of things done. That said, I'm pretty sure I'll be solving a lot more problems in Emacs than I ever did in any other single tool.

The second-generation acolytes turned a very pragmatic set of principles and practices into an esoteric scripture that could never meet the needs of reality. The "agile" moniker makes me cringe almost as much as "best practices". Both are cavalier substitutes for critical thinking, too often thrown about in the hopes that their mere mention will work magic. Ugh.

A real unfortunate side-effect of the backlash is the disregard some have for automated testing. Before I go any further, let me be the first to say that, like any other recommendation or practice, automated testing is not a panacea. I shouldn't have to say this. It's painfully obvious, like explicitly stating up front that I am against beating up defenseless children. Who is?

However, just because automated testing isn't a cure-all doesn't mean that:

• testing isn't worth doing
• testing isn't hard
• sometimes testing is so hard that it isn't worth doing

Note that last point. That there is what we in the biz call pragmatism. Yes kids, there are times when the cost of getting sufficient test-coverage is enormously expensive. Assuming that economics are part of your project somehow, you need to pay attention to this (which is why code coverage metrics can be dangerous in the hands of the nuance-free thinker).

Critics of automated testing like to point out that if developers write buggy code, how can they not write buggy tests? Assuming that it's as impossible to write perfect tests as it is to write perfect code, is there a point to writing the test? There is if the benefits you get from testing exceed the cost of developing and maintaing them.

Test-Driven Design/Behavior-Driven Design

The TDD/BDD philosophies are nearly as abstract and misunderstood as the entire concept of testing itself. So rather than parrot the original manifestos, let me break down the tangible benefits of driving your code with tests. The first misconception is the notion that you write all the tests first then you write all the code. This works almost as well as gathering all the requirements up front first and then writing all the code. I believe the technical term is "Waterfall". This is just dumb. Honestly folks, use your brains. Only a complete rookie would think that's a sensible practice to adopt.

Putting testing before writing code provides no magical effect that radically increases the quality of your code. Where testing does improve your code is when you use them as part of the code-building process. I like to use unit-tests as a form of mental scaffolding while I develop.

Breaking It Down

Any non-trivial problem needs to be broken down in some way. Few of us have the mental capacity to keep an entire solution in our heads. Even if we could, it generally doesn't last beyond the period in which the solution is developed. And even if you could do this, chances are few of your teammates would be able to. So at worst, breaking a problem down is a pro-social practice; at best it's a tool to ensure that you really understand what's going both now and in the future.

Generally when you break a problem down, you get into the business of assigning responsibilities to various components. This is a fractal pattern that starts at the highest-level architecture and repeats itself all the way down to the lowest-level classes, modules and functions of your system. It must be an inherent part of human-nature—we simply love putting things in boxes. Perhaps this is because that in doing so, we can hang a simple tag on a collection of related concepts as a form of shorthand. That shorthand allows us to talk about things in an efficient way. Imagine a design conversation where you couldn't name anything, but had to describe a piece over and over by its details. Yuck. So yeah, boxes are important.

So where does testing fit in? Testing allows you to write automated assertions about each of those boxes. It's like a logic game where you can start draw conclusions based on what you already know. It's like being able to say, "If this test is passing and that test is passing, the only thing that could torpedo this is if that doesn't work". I'll come back to this later when I talk about troubleshooting.

What unit-testing does not cover here, are the connections between the boxes. It should be obvious that it's entirely possible to have a totally sweet set of unit-tests that never fail that cover each piece of your solution in full and still have bugs or represent and incorrect solution.

This is because unit-tests only cover the boxes, not the connections.

Tests that cover the entire system are the best way to exercise the connections. They cover the boxes too, but they cover the integration (hence the name) of the boxes and connections. Since unit-tests can't cover the system as a whole, are they still worth writing? You bet. Keep reading…

Efficient Troubleshooting

When something goes wrong, you enter a different frame of mind from programming. Now you're in the detective business. You start with some scraps of information (stack traces, error messages, late-night pages) and with a combination of logic, guile and intuition (hopefully) find the root cause.

I could do an entire post on root-cause analysis, but let it suffice to say that most folks don't ask enough questions when troubleshooting. Often the conclusion people come to is an observation of a symptom rather than a thoughtful, system-aware diagnosis. In these situations it's a good idea to consider the 5 Whys Approach.

How do unit-tests help with troubleshooting? Like I said earlier, unit-tests are really good at testing the boxes, but not the connections. Let's assume that your unit-test accurately covers the box that you believe is the root of the issue you're seeing. Given that, there is a handful of conclusions you can draw: * The problem isn't really in this box * The problem is the connection coming into or going out of this box * Your understanding of the box is incorrect

That's it. It has to be one of those. Unit-tests allow you to confirm that your box is working correctly. If it's being invoked or interacted with in a way you didn't expect, it's generally a trivial exercise to write a test that does use your box in that way. If it passes then you probably need to keep looking.

If you're pretty confident that your box is good, then checking the connections is the next thing to look at. It's just as likely that the output of your box is being misused as the input to it. Integration tests are usually the best mechanism to catch this, but are generally much more expensive to write and maintain. For this reason they generally don't exhaust all of the possibilities. But with a real-world bug you now have a candidate for a new integration test. I can't stress enough the benefits of using bugs as opportunities to strengthen your test-suites. This is as true for integration tests as it is for unit-tests. It's generally not practical to write tests for every situation. So we use our intellect to make educated guesses that cover the most likely situations. When you observe funny behavior in your system, consider it a gift to your understanding. It's a golden opportunity to add a test for a real behavior, not a speculative one that may or may not occur.

So let's consider a world in which we have no tests. When something goes haywire you have to start from first principles every time. You have no mechanisms to get any leverage on the issue. Instead we have to rely on understanding the entire solution and holding it in our head at once. What do you think the chances are that we'll effectively troubleshoot an issue under these conditions?

Finding Smells

It is simply no fun to work on low-quality code day after day. As anyone who has every read Zen & The Art of Motorcycle Maintenance can attest, "quality" is a pretty squishy word. But in concrete terms, quality code means: * You enjoy working on it (or at least don't hate it) * You can expand its capabilities with reasonable effort * Somebody else can work on it too

I find the process of test-driving the development of code is a great opportunity to find "smells" in your code. These "smells" generally detract from code quality. The best catalog of these smells that I've encountered is the Refactoring book by Fowler and Beck. You don't need to read the book for a detailed description of the mechanics of refactoring, but it is worth reading to learn the terminology of "code smells".

At a minimum tests are the first "customers" of your code. If it's painful to test your code, you probably haven't factored it very well. Paying attention to this while your test-driving the development is a great way to keep your code high-quality. To really benefit from this, you need experience, a highly-developed sense of aesthetics and intuition. If you keep at it, you will gain these.

Note that I don't equate code quality with "success". There are plenty of systems out there that are successful that are not built on what I would call quality code. That's fine. I've made a conscious choice not to work on those systems. I spend an awful lot of my waking hours working on code—personally I don't want to waste that time being frustrated. I want a sense of accomplishment and satisfaction when I'm done for the day. Honestly, I'd trade success for quality code just about anytime. I expect to be doing this for a long time, so why not enjoy it?

Safety Net for Refactoring

When you need to extend or enhance a system, having automated tests serve as a great safety-net for refactoring. If you're confident in the efficacy of your unit-tests, you can be confident when you make changes and your tests still pass. You can refactor with a higher level of confidence when your tests keep passing. Of course there are no guarantees that you haven't introduced some kind of regression bug. But let's face it folks, the testing game is one of probabilities. It's too expensive to make sure you're right all of the time—instead shoot for being right most of the time.

There will always be times when something goes wrong and the only way to diagnose and fix it is to walk through the code painstakingly, line by line. No amount of testing will ever make that go away. My goal is to reduce that as much as possible. It's inevitable, but it doesn't have to be the norm.

Consider the case where you don't have any tests while you refactor. Your code/build/test/debug cycle gets a whole lot longer and more expensive. Because of that, you do it less often. Which means that when you do go through the cycle you have more things that can go wrong and more stuff you have to hold in your head. That increases both the likelihood that you will mis-diagnose a problem and that you will write more bad code to fix the latest bug.

I think this is how really twisted code is born. It almost reads like a blow-by-blow account of the developer's mounting frustration. The pace and phrasing of the code gets more and more frenetic: more odd comments, more TODOs and FIXMEs littered throughout, more long procedural stanzas. It becomes obvious that they were trying to get this done as quickly as possible, and that's generally not the path to quality.

Confidence

When it comes human-nature and software I'm a pessimist. I think we're really good at the intuitive parts and achieving the "ah-ha" moment. We are not good at holding giant complicated systems in our heads. There are people who can do that, but it's not the norm. For this reason, I think it's extremely foolish to build a software organization that relies on raw mental capacity. This definition of "smart" is one that can't scale and is unsustainable.

Automated testing is one teeny-tiny way we can offload the need for that capacity to a better long-term storage mechanism. While I write tests my mind is very focused on a particular part of the system. As I flesh out the tests I fill out my understanding of the problem-space at that time. Now that it's written down in code, I can safely leave it there and let my mind free up that space to do other things.

If I've written my tests well, I can get back into that focused frame of mind when I need to (e.g. for new features or bug-fixes). Without these, I have to spend a lot more time understanding a larger system just to get "up to speed". The tests can serve as an index into the design of the system. This is particularly effective when you have been paying attention to code smells and have been refactoring aggressively—your tests will map effectively to the underlying concepts of the system design.

This kind of confidence has a real tangible benefit. Without it I'm easily distracted by small details and dead-ends. As soon as I have more I have to worry about at once, I have less of a chance of succeeding at the task at hand in that moment. When I have such a thing in place, I have an instant force-multiplier in my hand. I can get something done. I don't have to repeat the entire experience of developing the code the first time. Instead I can leverage that experience to be more productive the next time I come back to that part of the system.

Conclusion

So there you have it. That's where I stand.

Is testing hard? Yes.

Does it catch everything? No.

Is it still worth doing? I think so.

This was causing all sorts of grief for me in some JavaScript I was trying to write. At first I tried to go with the flow and fix my JS, but it got really hacky really fast. So I went on a little source-code spelunking to figure how to make the problem go away.

In actionpack-2.1.0 there is a Proc attached to the ActionView::Base.field_error_proc class attribute. It's not documented in the RDoc, but this is also a writable attribute which means I can shut the damn thing up. Here's what it looks like in the original file, GEM_HOME/actionpack-2.1.0/lib/action_view/helpers/active_record_helper.rb:

require 'cgi'
    require 'action_view/helpers/form_helper'
    
    module ActionView
      class Base
        @@field_error_proc = Proc.new{ |html_tag, instance| "<div class=\"fieldWithErrors\">#{html_tag}</div>" }
        cattr_accessor :field_error_proc
      end
      ...
    end
    

My solution was to create a little file in config/initializers named field_errors.rb with this text:

ActionView::Base.field_error_proc = Proc.new do |html_tag, instance|
      html_tag
    end
    

Et voila! Just a simple pass-through with no more fancy-pants markup. Putting stuff like this in separate files in the initializers directory keeps your config and environment files from getting out of hand.

1.0.0 / 2008-09-19

• Added support for mapping dashes to underscores for flags
• Define Clip.hash.remainder as a singleton method instead of reopening Hash
• remainder works with Clip.hash now
• Reimplemented Clip.hash to use a parser.

Check out the docs at http://clip.rubyforge.org.

So I put in my proposal and got:

I wonder if they'll actually consider what I proposed. For that matter, I wonder if they'll consider any of the proposals users send them.

What is the bloody point?

It takes a brave author to give a book this title and keep it at 150 pages. The number of jokes about the proportionality of the "good parts" to the size of the book are endless. Be that as it may, when I saw this book at the Powell's stand at RailsConf '08, I figured anything written by Crockford on the subject of JavaScript was probably worth taking a look at.

Given the diminutive nature of the the book, I suspect the author was attempting to do for JavaScript what Kernighan and Richie did for C with their book. I found an abused first edition copy of that C book and it taught me more about C than any other book. It is a triumph of restrained, focused technical writing (for other examples check out any of Kent Beck's books). At times I felt that some chapters in "The Good Parts" were superfluous, but I imagine that they were included for completeness if nothing else. For example, the second-longest chapter in the book is about the grammar of the language. While important, it seems disproportionately large to rest of the contents.

If you have any exposure to JavaScript a good part of this book will be a review for you. However, there are three chapters worth reading for any JavaScript programmer. Chapter 3, Objects, discusses the properties of objects in JavaScript and covers everything from enumeration, to prototypes to attributes. Do you know why property enumeration is usually worthless as a debug tool? That's because of the prototype mechanism. If you use typeof filters and the hasOwnProperty method, you can significantly cut down the properties you dump on an object. I never knew that before.

Chapter 4, Functions, is the real money-maker of the book. This is the chapter that I put the most post-its in and will certainly be one I'll have to revisit. Here Crockford goes into the different ways functions can be invoked and what the consequences are of each invocation style. There is a lot of good functional programming material here. He also discusses how Functions are intimately tied to Objects. Take heed, thar be dragons!

Chapter 5, Inheritance, closes the loop on Functions and Objects. It explains the important differences between JavaScripts prototype-based inheritance model and the class-based model used in many other popular object-oriented languages. Like the previous chapter, this will be one that I'm sure I'll have to re-read in the future.

It's taken a long time, but JavaScript seems to finally be growing up a bit. Given how truly awful it was to use it in the Early Days on more than one browser, it's a bit surprising that it has come to dominate the web landscape. Now it seems to be finally getting (begrudging) respect from the rest of the software community. Unfortunately to be effective, you really need to learn some of the deep voodoo that, to me, is a bit counterintuitive. Is the prototype approach really all that innovative? I'm not sure. Arguing that it's better than inheritance is like saying that something is better than poking yourself in the eyes with a sharp stick—faint praise indeed. But I do love passing around functions as objects with complete abandon. When I finally grokked this part of JavaScript, things really clicked for me.

Is this book a must-read? No, probably not. If you're doing any serious JavaScript in the browser you're probably using one of the popular JavaScript frameworks out there that hide some of the yucky details of Function/Object interaction. But, lacking a decent language spec (and the ECMA spec sucks rocks), it's not a bad resource. It certainly won't take up much space in your bookshelf.

2.5 stars out of 5.

Alan Cooper's About Face is one of those pillars of UI/UX design, the reading of which is a rite of passage. I figured few books would be more appropriate as a capstone to my long list of design-oriented reads. It is nearly an institution in and of itself. Last night I turned the final page and ticked a pretty big 560-page book off of my reading list.

There was enough material in this book that I adopted a new habit of using small post-its to mark important passages. This was helpful not only in referring back to earlier material in the book, but also in cementing some of the concepts in my head. This practice was so useful I've started doing it with other books too.

You may have noticed that the post-its don't really kick in until about 2/5 of the way through the book (Chapter 10 to be specific). I'll be honest and admit that the first nine chapters felt like a rehash to me. That's not to say they are without value, but they are a bit long-winded and, at times, excruciating in detail.

So let me go ahead and get my beefs with the book out of the way. If you guessed that a book on its third edition that is nearly three inches thick would have some opinions you would be right. I don't have a problem with opinions—that's why I bought the book. But I did find the tone a bit much. Haughty, imperious, self-righteous and overbearing might be good descriptions too. There is a definite Moses-coming-down-from-the-mountain-with-fresh-tablets feel to this book. Mr. Cooper and his cohorts are here to set you straight. Don't let it turn you off. Think of these guys as that cranky-but-humorous guy you work with. He's a riot as long as you don't take what he says too seriously.

Secondly this is a long book. I'm not convinced it really needed to be this long. There's an awful lot of repetition spread throughout the chapters. All of that is nit-picking really. This was an incredibly informative book. Again, the number of stickies you see should give you some idea of the value that I got from it. There is a lot a person can learn here.

A big part of Cooper's philosophy is the focus on Goal-Oriented Design. The first part of the book spends a significant amount of ink describing the concepts of personas, goals and user research. As a developer I couldn't help but feel that a lot of this was Big Design Up Front. "Just let the arch-priests of interaction design get it all working and then hand it off to the developers", they say. Hmmm, my experience tells me otherwise. To be fair, the authors attempt to address in the afterword of the book. But still...

Once goal-oriented design is out of the way, the book picks up significantly by covering such concepts as the "excise" (tax) that systems place on humans, how people get into "flow" and how bad software can remove them from it or prevent it. One of my favorite little gems is a discussion of possible vs. probable. While it's possible that a user may want to choose one option over another, in cases where one option is disproportionately chosen over the other (the probable) use a default with an override. Think of how many times you've had to answer the same question over and over. It's pretty irritating, no?

Another great section is the discussion of metaphor. Of the book's numerous examples, the ones demonstrating bad metaphor really shine. General Magic's Magic Cap interface is an absolute nightmare of conflicting messages and unnecessary navigation. What do these icons mean? If I rub the lamp do I get three wishes? If I push the rubber stamp am I getting notarized or checking out a book? Just what the hell is going on here?

Cooper describes Magic Cap's failure as an over-reliance on global metaphor, where the system is essentially trapped in its slavish adherence to its metaphor. The rubber-stamp is there because real desks have rubber-stamps. But the need to be consistent with the desk metaphor weakens the interaction.

Cooper proposes idiomatic design as the alternative to metaphors. User interfaces have been around long enough that a large number of interactions have already been figured out. Users are already familiar with them and, generally, don't require additional ramp-up to recognize them. Obviously there are a lot of bad idioms out there too so, like anything in life, take that advice with some degree of moderation.

If there's one over-arching theme to the book it's that there are basically three groups of users you have to consider: beginners, intermediates and advanced users. The first and last groups are usually the smallest with the bulk of your user population consisting of perpetual intermediates. Beginners generally graduate quickly to intermediates. From there it's a much larger jump to advanced users. However, a lot of interfaces are often geared towards beginners. I would suspect (with little evidence to back this up) that a lot of this has to do with too much focus on customer-acquisition.

Without a means for potential intermediate users to shed their training wheels, users can get quickly frustrated. However you can't build a power-user-only application either. Very few users run the gauntlet from beginner all the way to the advanced user. So the trick to is spend the correct amount of effort on features that match the proportions on your users.

With this idea in mind the final eleven chapters of the book provide a fantastic, detailed look at how the principles described above apply to common visual idioms. For example, in the chapter devoted to menuing systems, the authors describe not as the sole interface, but a "pedagogic vector" for beginners. When combined with shortcut keys and accelerators, menus provide a way for beginners to graduate to intermediates and beyond.

Another important theme in the last section is just how far software really needs to come to meet users. The authors review example after example of "computer-first" design where the user seems be treated as a necessary irritant. One of my favorite passages in the entire book is on the topic of wizards: "Programmers like wizards because they get to treat users as peripheral devices."

Another chapter is devoted just to disk storage and how most of our idioms around disk storage should really be the computer's problem, not the users. At first this kind of talk seems like the crazy guy in Hyde Park, but after some reflection I think he's right. There is an awful lot of software that couldn't care less about the user. We can do better than this. We should do better than this.

One technical detail of the book that I just love is how they integrate images with captions. A particular pet peeve of mine is when images and captions seemed to be dropped into the text in willy-nilly fashion with little regard to the reader's flow. When figures that are referred to in the text aren't placed close to the text, I have to context switch and try not to lose my place. I either have to remember to look at the figure soon, or I have to switch to the figure immediately and then find my place back in the text. It drives me nuts. Whoever did the layout for About Face 3 obviously thought about the usability of the text. God bless you, whoever you may be.

While I'm pretty sure that I wouldn't want to have to talk to any of these guys at cocktail party (I doubt I'd get a word in edge-wise), they put together an incredibly informative book. I'm pretty sure I'll need those sticky notes in the future to revisit a number of concepts presented here.

5 out 5 stars.

So let's start with the domain model. I'm too spent at this point in the evening to port this to one of the standard examples. Instead I'll expose you to the domain of my particular problem. The app I'm working on is one that tracks (non-financial) lending transactions between two individuals. The parties involved, the item in question and when it's due are all tracked in the Transaction model (and transactions table). A Transaction has a number of states it walks through, using the acts_as_state_machine plugin. These state transitions are triggered by opaque-looking URLs that are sent via email to either party. These are one-time use actions that once consumed are no longer available. When an Action instance is created it also has a before_save callback that generates a unique ID (used in the URL) using Digest::SHA1.

So my plan was to have my Transaction class write one or more Action records for each possible action based on my state transitions. Take a look at the state diagram below:

I want to encapsulate the actual work to be performed within the Action instance the user invokes by following the link in their email. So my plan is to use STI to have different sub-classes of Action that operate on a transaction and march it forward to its next state polymorphically.

Now STI may appear to be total overkill for this problem, but here are my reasons for going this route:

• I want to have these opaque IDs written down somewhere to associate a specific action with a URL
• When the action is complete, I want to remove the record so it can't be performed again
• The state for a given Transaction can have more than one possible action. I want a separate for each action.

Whew. Okay, clear so far? So my initial code looked something like this:

require "digest/sha1"
    
    class Action < ActiveRecord::Base
    
      belongs_to :transaction
      before_save :create_guid
    
      def create_guid
        sha1 = Digest::SHA1.new
        sha1.update transaction_id.to_s
        sha1.update type.downcase
        sha1.update DateTime.to_s
        self.guid = sha1.hexdigest
      end
    end
    
    class ReturnAction < Action
      def execute
        transaction.return!
      end
    end
    
    class AbortAction < Action
      def execute
        transaction.abort!
      end
    end
    
    class DisputeAction < Action
      def execute
        transaction.abort!
      end
    end
    

It seemed like a good idea at the time, but the strange thing was that no matter which incantation I tried, I simply couldn't create a new Action instance and have it write a record to the database. This simply didn't work:

ReturnAction.create! :transaction_id => 1
    

There were no errors on the returned object. No exceptions were thrown. No queries to the database and certainly no insert statements executed. Just complete and utter silence. Out of desperation, as much as anything else, I removed the belongs_to declaration from the Action class and instead declared a has_many on the Transaction class. Voila! It worked like a champ.

After a bit of thought, the has_many association makes complete sense to me in the case where we want to create new Action instances for a particular Transaction. However, if you look in the code above, the execute methods of each sub-class are referring to a transaction object/method—which I no longer have. However I don't necessarily need the full-blown belongs_to association here. I can just fake the bits I want in the parent Action class like so:

class Action < ActiveRecord::Base
      def transaction
        @transaction ||= Transaction.find(self.transaction_id)
      end
    end
    

So none if this is particularly earth-shattering. Sorry folks, no great gems of philosophical wisdom today. Just one man's small accomplishment blown completely out of proportion.

Changes:

0.0.6 / 2008-07-10

• Fixed a bug with getting the 'remainder' when only flags are declared.

• http://clip.rubyforge.org

I am not a pro-war person. Anyone who reads good military history will often find that the best historians are quite anti-war. I had the opportunity to see Rick Atkinson speak in Seattle last winter as he was promoting his book, Day of Battle. He has written several titles about both WWII and Iraq and said that all of his books as anti-war and hopes his readers understand that. No historian has summed that notion up as well as John Keegan in his introduction to The History of Warfare.

To study war is not necessarily to glorify it. To be sure, a good bit of the military history bookshelf is filled with martial hagiography (see Ambrose, Stephen), but to those who study the history with more than a simple fascination of the technologies and armies there is a rich depth of human experience and tragedy to explore.

Like most men (and I'm sure that 99.9% of all military historians are male), I began with that very boyish fascination with all things violent and martial. As I grew older I realized that this was a silly childish interest. I either needed to stop reading about this subject, or I needed to understand it better. I chose the latter in an attempt to cull meaning from so many centuries of meaningless slaughter.

Humans go to war for a variety of reasons–most of them not good. Greed, hatred, and lust for power have fueled more than their fair share of conflict and tragedy. However that doesn't mean that I think all war is useless. There are some things worth fighting and dying for, but those moments are few and far between in human history. While I cannot condone military action in most of the cases in which it occurs, I believe that it is an inevitable part of the human experience. We were simply bred to beat each others brains out. This is a depressing and tragic conclusion, but I believe that human history is on my side when I make that statement.

So what's the point of reading military history? If all we are going to do is make the same mistakes over and over again what possible good can from studying the past? Despite my pessimistic outlook on the short-term chances of humanity straightening itself out, I think in long-term we may learn from our mistakes. Military history is nothing if not a study of mistakes. Yes the Hannibals, Napoleons and Pattons of history get the accolades, but there isn't nearly as much to learn from their successful campaigns. We need to look at the spectacular failures too.

For example, we can learn far more from the fateful decisions of both Napoleon and Hitler to campaign in the Russian winter. While the technical reason for their defeat was weather and logistics, the true cause of their downfall was hubris. It wasn't that neither failed to comprehend the risks, but rather they felt that their leadership and the elan of their troops transcended reality. Can anyone else think of a recent example where a leader ignored the facts on the ground in pursuit of their own personal policy? Hmm? Anyone?

Another fine example is the book I'm currently reading, Mr. Atkinson's aforementioned The Day of Battle. The Italian campaign is an oft-forgotten part of World War II. This is due in no small part because there was very little to celebrate in that campaign. No gutsy fortitude like Guadalcanal or Stalingrad, no tragedy like Poland or Saipan, just lots of dumb decisions that required a tremendous amount of cannon-fodder to achieve victory.

Few campaigns offer such a textbook example of group-think gone horribly wrong as the Allies in Italy during World War II. From Churchill and Roosevelt, to Eisenhower, to the theater commanders, everyone in the chain of command was eager to cast what they saw with their own eyes in terms of what they desired, not what the actual realities on the ground were. While we may not be in a war in our day-to-day work, each of us can certainly come up with examples where group-think led everyone to bad conclusions.

The other valuable outcome of studying military history to come to a better understanding of just what the cost of war truly is. Obviously anyone who has not experienced combat cannot imagine the horrors of that experience. I thank my lucky stars that I never had to go to war. But I feel that it's disingenuous to rail against the horrors of war without making an attempt to understand them.

I attended the University of Oregon and there are few places on earth that have a stronger innate anti-war bias. I have no problem with the pursuit of peace, but to do so in ignorance serves no one. It's vital that we all understand what the consequences of going to war are. I fear that the cavalier attitude with which the United States has prosecuted the Iraq War and insulated the public from the horrifying facts on the ground only serves to keep us ignorant of the true costs of war. This is not a game of cops and robbers–people are dying daily for questionable causes, to say nothing of the long-term political consequences. But I digress...

One final reward of the study of military history is to recognize how leadership within a corporate, in the purest sense of the word,  setting can work. I don't mean the famous generals such as Grant, Lee or Washington. Rather, a kind of leadership that few successfully execute consciously. Instead it is the Joshua Chamberlains of the world that provide insightful, meaningful case studies of what true leadership is. Most of us don't work in groups of one thousand or more, but instead in "squads" of ten or less people. Learning how Captains and Sergeants command, push and prod their troops and maintain esprit de corps is worth the time of anyone who is interested in exercising any form of leadership.

It's rare that at the end of a good military history read I don't feel the need to weep. Maybe I'm a sucker for heartache but I can't help but feel just a teeny bit smarter after turning the last page of a well-written piece of military history. So before parting, let me leave you with a short list of great military reads (in no particular order) that I've come across over the years:

• The Peloponnesian War — Thucydides
• The Civil War (Trilogy) — Shelby Foote
• At Dawn We Slept — Gordon Prange
• Just about anything by John Keegan
• The Best and the Brightest — David Halberstam
• We Were Soldiers Once...And Young — Harold G. Moore
• The Rise and Fall of the Great Powers — Paul Kennedy
• Diplomacy — Henry Kissinger

My setup is little-bit complicated, but I believe the following, stripped-down, recipe should work:

GEMDIR=$(gem env gemdir)
    OPEN=$(whence xdg-open || whence open)
    
    gemdoc() {
      ${OPEN} $GEMDIR/doc/`$(which ls) $GEMDIR/doc | grep $1 | sort | tail -1`/rdoc/index.html
    }
    
    _gemdocomplete() {
      reply=( `$(which ls) $GEMDIR/doc` )
    }
    
    compctl -K _gemdocomplete gemdoc
    

Update 6/25/08-10:30: Updated to work for both Penguins and Macs.

Today at Evri we're launching the beta version of our site. If you head to the home page and signup, you'll get yourself a free set of brand-new shiny credentials that will give you the keys to data-surfing heaven. The company blog post does a good job of highlighting what we have available, but for the truly lazy I'll give you the quick highlights.

First, is the home page which gives you a look at entities and their relations as we understand them currently. We start out with lists ranking the top people, places and things. In addition to popularity, you can also see who is rising and falling in popularity over time. All of these lists are clickable and enable our super-whizzy widget which provides a nice way of pivoting between entities, all the while getting related content.

This is one of my favorite parts of the product, as it's really easy to just get lost wandering around from link to link to see how things are related. It's like a big six-degrees-of-Kevin-Bacon game, except that you can do with just about anything. Part of that link-hopping experience is visiting specific pages about each entity.

Here you can find more detailed content about an entity. Currently these details comes from Wikipedia, but we anticipate adding several other specific sources of structured content. And of course, these pages link you to other pages, so between the hub-and-spoke visualization and the detail pages you can spend quite a bit of time just data-grazing. So take it for a spin and have some fun exploring! Give us your feedback (a link is at the top of each page) and let us know what you like and don't like. Most importantly, stay tuned. This is just the beginning for us, we have some pretty exciting stuff in the works and you won't want to miss out.

Things have been a bit nutty the last few days–as they are for any release–but it will be worth it for the satisfaction of finally lighting this candle.

Enjoy!

Do you see where this is going? No? How 'bout this?

Amen sisters and brothers. Now that's a properly-melted s'more.

The covers of both books were what initially piqued my interest. Both have very simple white covers. Unlike a lot of design books, this one isn't about showing off a bunch of pyrotechnics on the cover (see Jenifer Tidwell's Designing Interfaces which includes a colored version of O'Reilly's usual animal lithograph). I figured anyone willing to put such a sparse cover on the page was pretty confident about the content inside. I also really liked the fact that the form-factor of both books is smaller than the usual trade paperback and comes in at a very economic 200 pages, or about 1/4" thick.

Alright, I'll admit that I was taken in by the author's use of my favorite font, Gill Sans. I just love this font (it's the font this blog is set to if you don't override CSS) because it's clean and elegant with a minimum–or complete absence–of decorative fuss. Unlike the Pragmatic Programmers or O'Reilly the publisher, New Riders, doesn't seem enforce a particular font style for their books. Therefore I think it's safe to assume that the author made a conscious decision to use this font which, at a microscopic level, supports many of the notions of simplicity and cleanliness presented in the books.

Designing The Obvious

His first book, Designing the Obvious focuses on translating from what a user needs to creating workable screen-flows. While I've knocked out several smaller design books, I've been slowly making my way through Alan Cooper's seminal About Face in which the concept of goal-oriented design is introduced. Hoekman's book is the second text I've come across that offers a contrasting opinion on goal-oriented design the author calls task-oriented design. Whereas Cooper's approach is to begin design by understanding a user's goal within the larger context of their lives or career aspiration, task-oriented design is focused on more immediate desires.

A goal-oriented design might start with something like "Anna is a small-business owner who wants to balance career and family. She needs particular help with payroll for her small three-person company." A task-oriented design might start with "A user with a small-company must be able to setup employees with a minimum of fuss: perhaps just name, address and social security number". Hoekman even titles one of his chapters "Understand Users, Then Ignore Them".

Each chapter is tightly-focused on a single concept and few supporting ones. For example, the chapter titled "Turn Beginners Into Intermediates, Immediately" spends a thrifty thirty-five pages outlining the basic distribution of user expertise (hint: the big fat blob in the middle are the intermediate users) and then enumerating several concrete examples of how to serve the intermediates, how to get the beginners to immediates as quickly as possible, and how to keep the advanced users still engaged.

The ability of Hoekman to outline a concept and back it up with several concrete examples is the real strength of the book. This is not a patterns or recipe book. Similarly it's not a grand philosophical tome (see Cooper, above). Instead it's a very practical work intent on getting the ideas across, but leaving plenty of room for the reader to explore on their own.

The fact that he gets such a rich amount of information is such a small package is a testament to the author's well-thought, lean design approach.

5 out of 5 stars

Designing The Moment

Hoekman's second title focuses specifically on web application design. Unlike his previous book which is more philosophical and abstract, Designing The Moment is much more concrete. With thirty-one chapters spread out over 220 pages, each chapter is tightly-focused. Those chapters are split up into seven major sections.

The first section is titled Getting Oriented and focuses on getting the user oriented with your site. He delves into how users' eyes flow over a page, navigation, links and dealing with common web paradigms like tag clouds.

The second section, Learning, provides specifics about getting users "over the novice hump". This theme was an important one in his first book and here he offers several ways to teach your users about your site.

The third section, Searching, walks you through the pitfalls of search interface design. A common theme in both books is that of a poke-yoka, which means to fool-proof in Japanese. The term was originally derived from Toyota's Production System which was developed in the '80's (note: Toyota is currently kicking serious rear-end in the car biz). Here he uses the term poke-yoka device to mean any mechanism that will help prevent the user from hurting themselves. This is not about treating users as idiots but rather hiding ugly implementation details away from the users if at all possible. For example, you if need users to enter a phone number in a particular format, design a form that makes so that users enter phone numbers in that format. Don't just give them a text field and then complain when they don't get it right.

Moving on we get to fourth section, titled Diving In, where we really start to get into the nitty-gritty details of things like media player controls, form layout, wizards, and validation. This is the longest section of the book and meatiest. Again, Hoekman nails the concepts with well-chosen representative examples and solutions.

The fifth section, Participating focuses on the mechanics of features commonly associated with "social-networking" web applications. This chapter ranges from concrete implementation recommendations like how to build user profiles, to more abstract, strategic concepts like how to embrace user feedback and how to channel your most vocal users.

The sixth section, Managing Information, provides some tips on helping users digest your site's contents. Tips here include how to effectively use syndication, dealing with tags and folksonomies, where drag-and-drop is appropriate and dealing with system notifications.

The final section, Moving On, embraces I concept I first saw articulated in 37Signals' Getting Real . Don't build your apps as walled gardens where you make every attempt to keep your users from leaving. This ain't Vegas you aren't a casino. Yes, you want to give users another chance to reconsider and you certainly want to know why they're leaving, but don't be a jerk about it. This section offers some guidelines for parting ways with your users. I think to design something without this in mind is to the fact that not everyone is going to dig what you've built. Let them go easily and don't make things worse by making parting a painful experience.

I loved this book as much as Hoekman's first title. Both are handy references I'll keep nearby.

5 out 5 stars.

What the heck is "nice" anyway? The very language of that statement does nothing more than imply; it makes no assertions and offers no proof. A piece of candy is nice. So are flowers and a new house. But those things all require widely varying levels of planning, effort and cost. So "nice" isn't a very precise word and certainly fails us when we try to evaluate what goes into our product and what stays on the sidelines for further evaluation.

In a geek-heavy setting, such as my workplace, I often observe this phrase used as a desire to establish authority in a conversation. This happens when features are proposed not so much for their value, but as a way of showing how deep and nuanced the proposer's understanding of the domain is. For example in a brainstorming session we had recently about new visualizations for rich data structures, a lot of IWBNI ideas were proposed that were fairly baroque and hard to imagine being interesting to a wider audience. In this instance instead of focusing on the goal, making sense of a big pile of information, the exercise turned into a demonstration of various participants' grasp of market dynamics, web trends and cutting-edge features. You can imagine how useful the final set of ideas were.

Another issue with IWBNI features is that they are often very implementation-focused. For example if you are working on a web service that is consumed by other services, you might want to track usage statistics to generate a regular report to see who is using it. The real feature is tracking usage, but that original need can easily be obscured by an overly-specific implementation. Ideas for these features often emerge out of the system constraints that are a result of the system design, not necessarily a natural outcome of the problem domain. Differentiating between these two is probably one of the hardest things to do in any kind of design.

It's vital to differentiate between these because IWBNI features seem especially prone to calcifying the current implementation. Back to our web services example, let's say we implement this usage report, which dumps a text file every hour of usage statistics. We love that feature so much that we hook it up to our automated monitoring system as it seems like a nice way to monitor the "heartbeat" of the system. However, down the road we discover that we need a maintenance window that makes the file unavailable for a period of time that causes the monitoring system to alert us. Now we have a choice: we can patch up the monitoring to manage this exception, or we can re-think just how important the text-file interface is.

In this case I would argue that the text-file dump might instead be replaced with a simple web-request. When the system is in its maintenance mode, it could still answer questions about general availability (which is what we were originally interested in) without tying a more specific feature, usage, to monitoring. I don't think that making the monitoring script more sophisticated is the right answer. More complexity there means a higher likelihood of breaking and it spreads some very implementation-specific details to other parts of the system.

The features and attributes of a system can be viewed like walls in a house. Some are load-bearing and some are not. The load-bearing features are those that without which, the house would simply fall. In your applications these are features that are the very essence of the software. An application like Quicken has an awful lot of walls. The load-bearing features of Quicken are the ledgers and reconciliation process. Without these, the other features of Quicken are irrelevant.

However features like downloading transactions over the internet or viewing pie-charts are mostly decorative. This doesn't mean they are without worth, but they are not as essential as the load-bearing features. These could be removed and Quicken's essence would still be preserved. Quicken is certainly more useful with these features, but those aren't the features that generally drive users to use it. (NB: This is not to say Quicken is a well-designed application. But I'll save that for another post...)

You can move the decorative walls around to change the space of a room without major fuss. Moving a load-bearing wall is a fairly major operation and has a huge impact on the character of the house and requires extra planning so that the house doesn't collapse during the change. The danger of IWBNI is that it's easy to confuse these features with essential "load-bearing" ones. Worse yet, the compound of multiple IWBNI features can end up as a load-bearing walls that are difficult to move. Revisiting our web services example, if more features like the text file were piled on and external parties began to rely on these, it would become much more difficult to move these in the future. It's not difficult to imagine getting to a point where the original role of the web service is obscured by all of the other tangential bells and whistles.

Sometimes IWBNI features are user- or domain-driven. These seem like they might have a better chance of withstanding the litmus test. More often than not these ideas end up obscuring the core of the application, but these can be, relatively, easy to test with tools like mockups, user interviews and usability testing. In supporting services it's much harder to evaluate these features. How do we do usability testing for a web service? Does the variable for a request belong in the path or as a query parameter? How do we figure out what consumers want? This is tricky because in this case we're designing something by geeks, for geeks. This doesn't mean that it's okay to pile on a bunch of implementation details and stop thinking about separating our load-bearing features from our decorative ones. But it does mean that we have to be extra vigilant about the IWBNI features and view them with a particularly suspicious eye.

So I think this is the real trick–whether you are a visual designer, information architect or software developer–is to separate the essential from the decorative. Being able to sort features in this way gives you a chance to properly evaluate the cost/benefit tradeoffs. Without this I believe it is much more difficult to clearly see the value of a feature and its overall impact on the system.

So let me offer up a challenge: treat IWBNI as a codeword for something requiring exceptional scrutiny. When something is proposed as a IWBNI feature, regard it with a suspicious eye. When you feel yourself proposing a IWBNI feature, think long and hard about whether or not it is really "nice" or whether it might "essential" or "superfluous". And for goodness' sake, don't get hung up on your IWBNI features. If they're "nice" they probably aren't a core feature anyway. You're a smart, creative person and you'll have other ideas in the future.

And finally, let's remember the most common trait that nearly all IWBNI features share...

• Ya ain't gonna need it

Changes:

0.0.5 / 2008-06-12

• Removed sample_parser from bin (technomancy)
• fix a stupid bug causing an infinite loop for empty ARGV (technomancy)
• http://clip.rubyforge.org

After overcoming a few technical hurdles I got this blog moved over to Dreamhost. Things seem to be running well, but if you notice anything, let me know.

It sort of felt like the end of "T2" when Arnie willingly destroys himself in the forgery and signals a final thumbs-up before incineration. It was definitely time for that box to shutdown, but just a tiny bit sad just the same.

Changes:

0.0.4 / 2008-06-06

• Fixed typo in error message (thanks francois!)

0.0.3 / 2008-06-05

• Merged technomancy's patches for simple 1 LOC parsing -> hash

The Good:

I met a ton of people this year. Last year I went with a co-worker and we pretty much stuck together. This year I was on my own and made a concerted effort to just meet folks. By the end of the conference I couldn't go more than about fifteen minutes without running into someone I had previously met. I ordered a fresh batch of Moo cards before I left and I was hell-bent to hand as many out as I could. Just meeting people turned out to be my favorite part of the conference.

The Kent Beck address was fantastic. I've had the fortune earlier in my career to work at a company that had Kent come out and run XP workshops with us. Those experiences left a last impression on me (much like Chad Fowler expressed in the introduction) and so I was eager to hear his talk after seeing him on the roster. While his main content probably dragged for a bit, the Q & A ended with a bang. The answer he gave to the final question expressing a mixture of hope and concern brought the crowd to its feet. Go on, Kent.

There were a couple of presentations I went to that I though really knocked it out of the park. In some cases the material alone saved the day (in spite of the presenters) and in a few other cases the two came together nicely. I thought some of these presentations were particularly good:

• "Facebook Development and Performance with Rails" – Mike Mangino
• "The Launch: Do's and Don'ts of Real-life Deploys" – Chris Wanstrath
• "Assembling Pages Last: Edge Caching, ESI & Rails" – Aaron Batalion
• "Skynet: A Ruby Map/Reduce Framework" – Adam Pisoni
• "Vertebra" – Ezra Zygmuntowicz
• "Advanced ActiveRecord Techniques: Best Practice Refactoring" – Chad Pytel

The Bad:

Sadly, a number of the presentations were pretty lacking. Now I think presenting is just plain hard and very few people are good at it. Keynote helps a bit, but really it's a crutch for people who don't have good public speaking skills (which I'm not necessarily claiming I have). Really exceptional content can help overcome the stylistic short-comings of a particular speaker, but I think that's rare. I think it's pretty easy to lose an audience quickly if your material can't shine in the way you present it.

Since I have another year of Ruby and Rails experience under my belt, many more of the talks just really didn't do anything for me. That's why I'm psyched that the RailsConf team has decided to incorporate Caboose Conf as the "hallway track" in next year's meeting. I think that's the track I'll be taking next year. I did a lot of ingesting this year, next year I'd really like to produce more.

The Style Report:

Apparently the black t-shirt is king in the Rails community. Something like 99% of all attendees had black t-shirts on. Of the free t-shifts to be had, the GitHub tees were the only ones that weren't black. All of the others were red on black. So apparently the new black is, well, black. I'm hoping that nuclear orange makes a comeback next year. All-black is just a little too Depeche Mode for me.

Portland:

Portland deserves special mention because I just flat-out love that city. Besides Seattle, it's probably the only other city I would choose to live in. One of the highlights of experiencing the local flair was getting involved in the 1000-person "doughnut march" held by Portland's beloved Voodoo Doughnuts. Somehow they convinced officials to get a parade permit and police escort as they crossed from their Pearl District digs to their new shop in East PDX. It was a very "Portland" experience with a whole crowd of folks letting their freak-flags fly high.

The big treat at the end was the bacon maple-bar. I'm not kidding folks, this is real and it's freakin' brilliant. So hear this Chad Fowler and the rest of the RailsConf committee, please don't move this to Vegas! Besides, can you imagine putting a bunch of hygienically-challenged nerds in 100-degree heat in a desert? That is simply not a good idea.

As you might suspect, 1Password is a master-password application. You can easily generate hard-to-crack passwords (like, '81239jsdfj912jlksdf8981'), but only have to remember one master-password. This in and of itself isn't particularly interesting (though the password-generation feature is really well done). What makes 1Password so compelling is how well it is integrated into your online activities. 1Password works with just about every kind of web browser you can imagine on a Mac.

On Firefox, 1Password can install a toolbar that puts all of 1Password's features right there. When you navigate to a site for which you stored a password, you can automatically fill out a form with the click of a button. If you haven't re-authorized with 1Password in a bit (often after waking up my laptop) you have to give your master password again (nice touch boys).

Also whenever you submit a page that 1Password thinks is auth-related, it will offer to remember these credentials for you. When you update your credentials on a site, the app will ask if you want to update your existing credentials. That is absolutely killer. Clearly these guys understand the achilles' heel to most security systems--people are lazy and are the weakest link. Having such smooth, inline integration makes it difficult for even the lazy user to subvert their own security.

1Password doesn't just live in your browser, it's also a nice stand-alone application. You can easily modify and update your secure information as well as import and export it (a handy way for me to sync my work and home machines).

You're not just limited to passwords either. The app also comes with a notion of identities where you can specify the contact information you might need at various web sites. For myself, I keep both "work" and "personal" profiles. Whenever I have to sign up for something I can simply click the profile I want to use and 1Password fills out as many of the relevant form fields as it can. It also has a "wallet" feature where you can stuff various other secure information like credit card numbers (including the verification number on the back), AWS keys and host of other things. Very very very cool.

Perhaps the most supremely cool feature of 1Password is the ability to sync encrypted javascript bookmarklets with your iPhone that gives you access to those same secret items on your iPhone! 1Password creates two special bookmarklets (synced via iTunes through your Safari bookmarks). One displays your passwords, the other will fill out web forms with with the selected credentials. Both require you to enter a password to unlock the bookmarklet, which you set when you export the bookmarklets from the application. Being able to get that secure information away from my computer makes sure that I don't set simple passwords for iPhone access just because I can't get to my secret stuff. Well done, boys, well done.

The features of this app alone are enough to have sold me. But a good part of the impression this tool has made is how nicely it's been done: no sharp corners, wonderful usability, smooth integration. Even the experience of entering the license key is cool. Your license is an image file (no doubt with some extra steganographic goodies embedded) that you simply drag to a target area. No funny numbers to type. How freakin' cool is that?

I've got a whole 'nuther post cooking up about my master-list of indispensable nerd-tools, but I felt that this app was worth a standalone post. If you're on a Mac, do yourself a favor and check this app out.

P.S. No, I wasn't paid a dime to write this.

At work, we're getting very close to start unleashing our product on the world and so I was doing a little demo for my mom. At first this started to feel like other times when I've showed her things I've worked and she'll focus on some seemingly insignificant detail or re-imagines the application in terms that seem very specific to her world. I felt my usual generation-gap defenses warming up and I started to wrap up the demo. No point in going any further, right?

But after a few more moments I was able to demonstrate enough compelling features that she was engaged. More importantly I became engaged in her reaction to the system. The barrier between us where each side couldn't really grok the other seemed to fall. I was able to have a better understanding of how she might look at it, instead of how I've been looking at for the past year and a half.

So perhaps it's trite and obvious to state that one can benefit from giving yourself the time and patience to understand your users. But I think it's especially true when dealing with other generations. If you can come up with something that crosses those barriers like my impromptu demo did, by George you're probably onto something.

Changes:

0.0.2 / 2008-05-20

• Cleaned up README
• Added support for late-binding option processing with blocks

Why S3? Well, something as important as backing up my music (which hasn't come from a CD in about three years) means I need multiple copies. I certainly plan on getting a local backup happening soon, but having one out "in the cloud" provides a little extra comfort if things go totally haywire. Of course none of this would be possible if S3 weren't so darn cheap. By my estimates my 50GB music collection will cost me about $6/mo to backup. That's pretty cheap peace of mind if you ask me.

I did the backup using excellent S3Sync utility. Every S3 tool has its own way of mapping a filesystem to S3's bucket system. I thought about using JungleDisk in headless mode, but that turned out to be more complicated than I was willing to deal with. The nice thing about S3sync is that it maps files to your buckets in a way that is compatible with the S3Fox extension for Firefox. Most excellent.

Running over SSL with my crippled Comcast connection it took about a full week to push all of that data out to S3. Thank goodness for Gnu's screen utility which hosted the entire sync process. That's right kids, I did this all with a single invocation of s3sync.rb that ran for a solid week. Boo yah. Total upload cost was $3.58. Not too freakin' bad.

Introducing Clip...

Checkout the details at http://clip.rubyforge.org, or just dive right in and install that sucker as a gem.

Changes:

0.0.1 / 2008-04-10

• Initial release for y'all to throw rotten veggies at.

Always run rsync with --verbose and --dry-run to make sure it's doing what you think it's doing

So I decided it was time to re-write the script to support this. While you can do command-line options with bash, it quickly gets kinda oogy, so I fell back on Ruby instead. I've also collapsed the synchronizing down into a single script--one that goes both ways. So without further ado, you can download the script here. This should work with a stock Ruby install, no special gems required.

Update 4/8/2008: Okay, I still don't know what the hell I'm doing. There is a bug with this script in that if you create a new file locally then try to sync from S3, your new file will get obliterated. Well guess what kids? Synchronization is hard. I've been noodling on a variety of hacks to get around this but none are terribly satisfying. Anyway, my Golden Rule (see above) still stands: make sure you test the thing out before you run it "live".

In short, here's what you can do:

c = Filtering::NaiveBayes.new
    
    c.train("Nobody owns the water", :good)
    c.train("the quick rabbit jumps fences", :good)
    c.train("buy pharmaceuticals now", :bad)
    c.train("make quick money at the online casino", :bad)
    c.train("the quick brown fox jumps", :good)
    
    c.prob("quick rabbit", :good)  #=> ~ 0.156
    c.prob("quick rabbit", :bad)   #=> ~ 0.050`
    

Here we create a new NaiveBayes classifier, train it with some text and then query it with other text. Nifty eh? There is another classifier included in the package called Fisher, which has a slightly more clever classification algorithm.

Both of these default to in-memory storage of classification data. You can override it by using the built-in ActiveRecord persistence adapter like so:

ar_config = Filtering::Persistence::ActiveRecordAdapter(
      :adapter => "sqlite3",
      :database => "mydb.sqlite3"
      :timeout => 5000
    )
    c = Filtering::NaiveBayes(ar_config)`
    

Finally, there's an executable in the 'bin' directory where you can interactively classify RSS feeds using either of the classifiers or persistence mechanisms provided. This relies on the feed_tools gem.

So there it is. The rest of the pci4r team should be spooling up soon and hopefully we'll make some more progress. Stay tuned...

At work I have a MacBook Pro hooked up to a large Dell flat-screen LCD monitor. The screen real estate is fantastic, but it has a weird effect on my terminal program. Here's a snapshot of what the Monaco font looks like when I start iTerm (or Terminal.app) while attached to the Dell:

Here's a screenshot of what iTerm looks like when I launch it without the Dell attached:

See how much thicker the letters are in the second shot? I much prefer the latter setup because it's easier on my eyes. Also I find the variation in the thickness of the lines (probably a result of anti-aliasing with the Dell) in the first screenshot distracting. But that's just my font-related OCD kicking in. It bugs me enough that if I have to start iTerm again, I'll unplug the Dell, wait for my Mac to figure out I have one screen, relaunch iTerm and then plug the monitor back in. Yup, it matters that much to me.

It seems stereotypical that dudes who write Ruby on Macs love big fonts. Meeting other Ruby dudes with Macs was certainly my first exposure to terminals set to 18-point type. Before then I felt like 14 was pushing it. But then I realized that there really isn't that much I can look at at once. I used to be one of those guys that would get four terminals going simultaneously in 10-point type. Just look at me, I'd think to myself, I am soooooo productive. Now I'm on the other side of the fence. I'm a big-font guy.

In part this is because I know that I can get distracted too easily. Having a bunch of stuff open at once only opens me up to more opportunities for something to steal my attention. These days I really don't need to exacerbate my ADD so cutting down on what I have to look at is generally a good thing. I've actually returned to using virtual desktops just to partition my running applications. Oh yeah, I turn off all those notifications too. It was fun for awhile, but now it's just irritating.

I like to keep the fonts pretty big. I don't like reading from a screen nearly as much as I like reading from paper. I can feel a sort of mental fatigue set in after reading too much online. One way I've found to combat that fatigue is to simply increase the font-size. Honestly I can't read five paragraphs at once...I can read only read one so I really don't need to cram a bunch in a single space.

I think I'm not alone in my preference for big bold, stripped-down fonts. I've seen an increasing number of web applications that go with a real stark, big-font look. One of my favorites is GitHub:

I think it's pretty clear what you're supposed to do here. Very little extraneous BS, just direct to the point. Give them a login and a password, they'll give you GitHub. It couldn't possibly be any simpler. Oftentimes you see the login box relegated to a corner of the screen with the remaining 80-90% devoted to some kind of market-ese with stock photos of trim, athletic, happy people who are ostensibly using the same product. What a total waste of space.

Size isn't my only bone to pick with fonts. I'm also quite particular about which faces I like. In general I'm not a big fan of serif fonts. I find the clutter of the extra lines distracting, so I generally stick to sans-serif fonts as much as possible. Among sans-serif fonts I have two favorites: Monaco for fixed-width and Gill Sans for variable width.

Fixed-width fonts are popular in programming because the code just lays out better to our eyes. It could be that with things like blocks and indentation, programmesr have a natural tendency to scan code horizontally and vertically in rows and columns. Vertical scanning would be much more problematic with a variable-width font.

However for general reading, I like Gill Sans as a variable-width font. I like it so much that whenever I setup a new desktop, I go fiddle with the font settings in Firefox to get 'em the way I like 'em. I even go so far as to trump a site's stylesheets just to keep my font. Sometimes this results in pretty weird looking pages, but more often than not it turns out okay.

However, I realize that not everyone likes the same font I do. Because I do some web development I need to be able to view sites the same way as the poor unwashed masses view them. For that I take advantage of Firefox's profiles which allow me to have my font-fascist settings for general browsing and factory default font-settings for testing.

So there it is. I like big fonts and I'm not ashamed to admit it. Sure it might look like I'm reading the code-equivalent of the large-print section from the library. But hey, I still have great vision and you small-font people aren't processing any more at once than I am.

Dashboards are one of those phrases that the synergy-loving, tassle-loafer, khaki-loving business-types really like. I was worried at first that this book would be targeted more toward the executive officer crowd and less about helping designers. The first two chapters didn't ease my concerns with lots of discussion around designing for executive goals and summaries. However once we got past the introductory chapters, the book really picks up...

Chapter 3: Common Mistakes

This chapter highlights a litany of common dashboard design mistakes. These aren't just limited to visual tips, but also includes important concepts like providing context through comparison and scaling displayed data appropriately. The examples given here are fairly egregious examples of bad dashboard design, but acclimate the reader to critical review.

Chapter 4: Tapping Into The Power of Visual Perception

This is where the fun begins as you learn some of the science behind visual perception. Few describes how shape, position, color, hue and encapsulating marks all affect how we process and perceive data visually. This is one of three chapters that a designer will return to again and again for review.

Chapter 5: Eloquence Through Simplicity

Here we head to a particularly Tufte-ian territory with the consistent refrains of "maximizing data ink" and "removing non-data marks". While the words make sense, the concepts are driven home with some great examples. My favorite part of this chapter was the rectangle depicting which screen regions get the most attention and why. After you see it, it's obvious and yet you'll easily be able to recall many examples you've seen that violate these principles.

Chapter 6: Effective Dashboard Display Media

The party continues with a detailed survey of the most common graphical visual idioms. The common chart styles like line plots, bar graphs and scatter plots are explored as well as the more exotic ones such as radar graphs, stem-plots and even strategic icon placement. This chapter serves as a great foundation for the final chapters where you will use your newly-acquired visual vocabulary to perform a detailed critique of a number of sample dashboards. Oh and if you take nothing else from this book, remember this: pie-charts are useless.

Chapter 7: Designing for Usability

This is the shortest chapter of the book, but the two great takeaways from this chapter are encouraging useful comparisons and avoiding senseless ones. If unrelated data-sets are placed too closely or use the same visual representation, it may encourage meaning comparison. Conversely using different styles for data that should be compared simply makes the user's job harder.

Few dives into a few tips around making an esthetically pleasing dashboard without resorting to decorative gew-gaws that clutter the display. In particular things such as using a muted color palette, designing for high resolution and picking the right fonts (a particular sticking point for me) are all ways to be informative without distracting the user.

Chapter 8: Putting It All Together

This is easily the best part of the book. However I wouldn't suggest skipping the other chapters in a effort to get to the payoff. I think the reader would best be served by reading this book in sequential order because the principles and concepts covered in the earlier chapters help to give the reader a visual vocabulary for what comes in this chapter.

The begins by showing a dashboard the author designed designed for a competition designing dashboards for a mock business. The dashboard is concise, dense without being cluttered and draws focus to important items quickly. In contrast the author shows eight other dashboards that contestants submitted for the same mock business. Each one has a number of defects, some easy to find and others less so. It's definitely worth studying these dashboards with a careful eye prior to reading the author's criticism.

The chapter ends with three more dashboards targeted to other audiences: the CIO, tele-sales and marketing. Each shows off the other motifs highlighted in earlier chapters. However they all have a consistent style--one that is clear, concise, dense but uncluttered. The prospective designer would be well-advised to spend some quality time studying these examples to distill them into principles for their own designs. This chapter really caps the book off well.

Finally...

Information Dashboard Design is a well-written book full of useful information. The high-level principles are well-described and the catalog of details motifs make this book worth having.

4 out of 5 stars.

I'm waxing poetically because I just acquired my first Slicehost account for a side-project I'm working on. I've dozens upon dozens of Linux installs over the years, why should this be special? Perhaps it's worth a trip down memory-lane first...

If you discount those early years writing BASIC for the Apple II and TRS-80, I've been living Unix/Linux longer than I've been programming. In that time I've gotten rather particular about my distros and how I like to configure them. I first cut my teeth on HP-UX working at a wireless telco. Some co-workers there introduced me to Linux which like having an equivalent HP-UX-like interface on a 486 under my desk at home.

Like many, my first distro was the tower of floppies known as Slackware. Ah, the good old days. When setting up X might have entailed destroying your monitor if you didn't get the parameters right. When I started doing actual development I moved over to Red Hat because it was the most polished distro at the time. After that I had a brief flirtation with SuSE, but found the configuration frustrating although I did have a major success getting dial-up internet working via number of arcane Hayes modem commands and some network scripts I found on the web.

Then I discovered Mandrake, which I stuck with for a number of years. It was Red Hat-based, but had a much better installer and better package manager. In the end though, even the package management improvements could not overcome the inherent flaws of RPM. On countless occasions I would upgrade and the entire sweater of package dependencies would unravel and suddenly I need to upgrade to a new version of glibc just to get a decent RSS reader working in KDE. Ugh.

It was then that my most Linux-geeky friend turned me on to Gentoo. You will learn more about linux kernels and configuration running Gentoo than you ever imagined. Mind you, I didn't get into Gentoo because I wanted to hyper-optimize my install of emacs. I switched because Gentoo's package management and configuration beat the snot out of Mandrake and I found the ebuild system rather elegant.

Unfortunately Gentoo took a tool on my patience with Linux on the desktop. Gentoo required far more care and feeding that I could give it. I spent far more time building and re-building my system than actually doing anything with it. I will point out though, that doing a bare-metal Gentoo install all the way to a full-blown desktop manager like KDE or Gnome has to be about the best hardware burn-in test there is.

So then I hopped on the Ubuntu band-wagon. While there are far too many ways to install packages (apt-get? dpkg? adept? wirble?), the Debian package management is a nice compromise between pre-built package systems like RPM and configurability (a la Gentoo). So that's what I've put on my new Slicehost host. It's well-documented, has tremendous community support behind and is much more up to date than it's older, conservative sibling, Debian.

So back to Slicehost. So far the experience has been tremendous. I would say that less than a minute elapsed between the time I decided to give them a credit card number and when I got a console login. They have several pre-built server images you can use to provision your slice. And now it's mine...all mine. I get to set it up just the way I like.

So what's in the soup, you may ask?

Web Server: nginx

I've done a lot of Apache over the years and it's hard to overestimate the impact that it has had on the web. However, it's configuration has become nightmarish. In this case my needs are pretty narrow, so I'll go with something easier to work with. Plus it has a rad name.

App Server: Rails!

I hope I never have to do Java-based web development again. As far as dynamic language go, I've thrown my hat in with Ruby rather than Python, although I hear great things about Django and I'd like to play with it. There are certainly other viable Ruby web frameworks (Merb, Camping, ...) but for this project I'm more interested in getting things done than learning a new web-stack.

Database Server: MySQL

I really really really wanted to do this with Postgres since it has just about the best command-line tool I've seen for any RDBMS. I also find the permission model in MySQL to be byzantine and difficult to debug. But alas, scaling out to multiple nodes with read/write master/slave and replication is much more do-able in MySQL than Postgres. There's a ton of existing literature out there on it and I've had experience doing it.

Mail: Postfix

Eventually this app will need to support incoming mail so simple MTA's like exim or esmtp weren't going to cut it. Since I'm deathly afraid of Sendmail, I figured I'd go with Postfix. This is probably the tool I know the least about right now, but it should be interesting to learn.

Monitoring: monit or god

I've used other commercial and open-source monitoring tools and they've all felt really heavy-weight and invasive. I don't so much mean "heavy-weight" in a CPU-sense, but in a process and procedure sense. Looking at the sample configurations for both god and monit put a smile on my face. god looks particularly interesting because it's in Ruby.

Text Search: solr, ferret, sphinx

I've spent a little time looking at all of these solutions to supported inverted-index, full-text search support. I'm not 100% sure where (or if) this app will need full-text search so this area is a little bit hazy. I'm not committed to a particular implementation technology here (i.e. running Solr which is in Java is totally viable).

Message Queue: .....

I'm more sure that we'll eventually need to have asynchronous, queue-based work processing to handle things like image resizing and storage or document generation. Like the text search, running something like a Java-based message queue (like ActiveMQ) is entirely possible. A major piece of the selection criteria will be based on ease of integration.

So we'll see how this goes. I run my own Linux box at home to host this blog, a Subversion repo and couple of other things. It's nice to have a virtual equivalent sitting on much fatter pipes. I'm excited to see where this goes.

Yesterday I checked out a site that plays in the same space as a project I'm working on. These sorts of things are bit like going to the dentist, you don't necessarily look forward to it, but you do it because it's good for you. It's not that I'm trying to ignore what is out there, but for a side-project there is a certain bliss in being ignorant of what else is out there--especially when there really isn't any money at stake. But I was a good boy and checked it out. I went in fearing the worst--that these guys would have nailed the concept and the interface, and there was simply no reason to put any more time on our version. After about fifteen minutes I was pretty convinced that we could come up with a more compelling user experience.

Like mom always said, if you don't have anything nice to say, say it anyway--just don't name names. So let me try to provide some detail on what I found without tipping my hand. This app had all the right rounded-corner-drop-shadow-muted-blue patina that any self-respecting Web 2.0 app should. But digging a bit under the surface revealed a pretty face on what was essentially a Windows for Workgroups-era application mentality. The user flow felt like it was driven from the underlying relational data model, not necessarily from sensible use cases. As I filled out wizard form after wizard form, I couldn't help imagining each one mapping to a particular table in a database. In addition, each form had several tabs with a mix of required and optional features to be identified by the user. The tabs were clearly meant to further sub-categorize the input, however because of the limited real-estate in a tab, only single-word descriptions were available. I had to click on each tab to see if I needed fill anything out in there or not. Too much clicking to satisfy the data model, not enough payoff for the user to keep it up.

Then I ran into something called "templates". I can't help but wonder if templating concepts aren't a smell rather than a feature. In the case of this application, it wasn't really clear what the benefits of templates would be. Normally templates are there to avoid repeating work and providing a model of re-use. Looking at this "feature", I couldn't help but wonder why I should care. If templates are so damn important, do I really want to use an app in which templates are a necessity? Again, too much work, no clear payoff. Humans serve the machine. Bah!

So why would somebody put something like templates in? The answer is re-use. Software developers simply love re-use wherever they can get it. I can imagine the developers and product team sitting around reviewing this complicated system they've built and finally having to address the fact that the complexity makes it hard to use. Templates to the rescue! We can reduce the user's work-load with reuse! Ugh. It's only an economic win if I really care to continue the app. Otherwise the real problem lies in the original interaction model.

The Golden Age of Perfect Software is nowhere on the horizon so building software will continue to be a type of gruesome sausage-making for the foreseeable future. One of the great lies about software development is the false economics of reuse. The theory goes like this: if applications can be built by assembling more-powerful, higher-level components together, the overall cost of software development should decrease. At face value this theory makes a lot of sense. I'm sure many of us can think of time we have wasted (or seen wasted) on building rounder wheels. Why should anyone spend time working on a library of sorting algorithms when for 99.999% of all cases the sorting routines available with your platform are adequate?

The problem is that the component-based thinking doesn't always match the needs of the application. Sure we may have built it more cheaply than if we had written it from scratch, but we have to ask ourselves how much did we compromise in that effort? Software is a notoriously tricky business and the number of failures greatly outweigh the number of successes. Google wasn't built in one shot. Heck, the first version of Google Reader was so badly panned, they threw it away and wrote another one. Apple is on the tenth major version of the OS software and fifth major revision within the latest family. Let's not kid ourselves, we're much more likely to get it wrong than get it right. So let's just assume that it's possible that none of the available, reusable components is going to suffice for your needs. That doesn't have to be your starting assumption, but we have to keep the possibility in the back of our minds.

This leads to a second point I'd like to make about re-use. Many UI toolkits come with a pretty rich array of widgets that are used to build nearly every application out there. This is true of native clients as it is of web applications. If our goal is to minimize the conceptual distance between the user's mental model of the application (remember, the developer's mental model is almost irrelevant) and what they need to do to meet that model, then restricting ourselves only to what the widget palette offers will be fraught with compromise. Developers may see applications built as collections of widgets, but users often don't. It's quite possible that using the "standard" widgets for your application greatly increases the mental distance between thinking and doing for the user. When this happens the user is serving the application--not the other way around. Uh oh. Wrong turn. Back up. Try again.

Don't misunderstand me. I'm not saying that every application has to be written from the ground up because there is no hope in getting the interaction right by reusing off-the-shelf components. That's ridiculous. We'd never get anything done if we approached our apps like that. Rather, my point is that we can't necessarily limit ourselves to the widget sets and conventions of our platforms. In doing so we may leave the user out in the cold. However, visual or interactive exceptionalism for the sake of differentiation invites similar risk. Dressing up your app to stand out is akin to putting lipstick on a pig. If you app is compelling, dressing it up will be unnecessary.

Let's return to our anonymous whipping boy...er, sample web site. Not only did the wizards and multiple-layers of object hierarchy force me to learn the tool's model (one I don't care about), but the expression of those wizards (the widgets) similarly required me (the user) to play along, rather than getting me to the payoff. Some applications are very sophisticated and will require more of the user, but that's not the case for this one. Indeed the selling point is how easy it is to use and how user-focused it really was! Boo, hiss. F-.

So while the wizards had a very consistent and recognizable look and feel to them, they did a horrible job of getting me to where I wanted to be. I can't help but feel that this application was built with a primary focus on a relational data model and component reuse. This ended up with fat proxies between me and what I wanted to get done--an increase in the mental distance between thinking and doing. This is the real shame. Increasing that distance is the primal sin of application design.

One final note: a lot of the inspiration for this post came from a fantastic post by Cathy Shive on, what she terms, "Administrative Debris". I interpreted "debris" as that which increases this conceptual distance I keep harping on. Her post, unlike mine, provides a lot more visual examples to get the point across. I would highly recommend reading her article to get another angle on these ideas.

This is all I walked out of the local Apple Store with today. No unnecessary bag. No receipt I have to dispose of securely. Just the product I wanted and a friendly little "Thank you" sticker. If all stores were like this, I might actually like to shop.

The local Apple Store also switched over to a cool new portable POS system where just about any sales associate can swipe your credit-card and allow you to make the purchase there. They must have been paying attention to customer complaints because before the remodel, the lines to the cash register were absolute heinous.

This was fantastic. I walked in, a sales associated asked what I was looking for. I told her and she immediately directed me to the available products. I picked one and she took to me another guy who had one of these little POS thingies. I gave him my card, he asked if I wanted my receipt e-mailed to me and I was out the door in less than two minutes. In the time it took my wife to pick out a wedding present I walked a block to the Apple Store, completed this transaction and made it back to where she was before she was ready to go. Fantastic.

Our particular location covered six precincts. I wish I had taken a snapshot of the precinct signup--it was utter chaos. I have no idea if there were any Fire Marshalls in the crowd, but they wouldn't have been happy with what they saw. However, I did manage to get a snapshot of our precinct's meeting:

This picture doesn't really do justice to the size of the crowd. Our precinct had over sixty people register (though not all of them stayed for the meeting). This was my first Caucus and it was certainly an interesting and eye-opening experience. It was completely disorganized--our precinct head wasn't there so we had to elect one. What we lacked in direction was made up for in earnest desire to pick the best candidate for the Democrats.

Like most of Washington State Democrats, our precinct was overwhelmingly for Barack Obama. That's not to say that it was a room full of Hilary-haters. Indeed many Obama supporters voiced their pledge to support Clinton if she wins the nomination. However two themes emerged as people pled the case of Obama.

The first was that the sense of change that Senator Obama would bring to the nation is one that is desperately needed. The second is that having Senator Clinton in the Whitehouse would lead to four years of fractious fighting with the Republicans which would prevent the nation from addressing the stack of dire issues we face.

After we voted a couple of times, the final tally was held. We then split into the two camps. The Clinton camp needed to select one delegate to send to the city caucus and the Obama camp needed four delegates. At this point my wife and daughter left because Audrey, bless her heart, hung in like a trooper but was starting to fade. A Tootsie-pop can only take a kid so far. I stuck around for what I thought would be a lengthy delegate-selection process but what, in fact, only took about ninety seconds to decide. We selected four delegates and four backups in record time which sort of made up for the previous hour's worth of bumbling.

We'll see how the race turns out as this primary/caucus season unfolds, but it was certainly exciting to participate in the process and feel like it mattered.

Here then, is an amateur's critique of the various infographics I tripped across. What I was looking for was something that would tell me:

• Margin of victory for each candidate
• Percentage of precincts reporting
• The number of delegates available
• The number of delegates each candidate won
• Clear indication of races that have finished, and those that have not.