Zed's CUSEC talk hopefully inspired a lot of people to take a good look at the work conditions they're either in already (like me) or will be in soon. At my most recent job I really gave my all. I spent all my time thinking about the job, doing tons of (paid) overtime and basically making it my life. It was fun. I was interested. Then I got tired of it all and quit. I don't intend on ever stopping being passionate about my job because I don't think I could stand doing something for the majority of my waking hours and not enjoying it. I'm lucky because I know people will pay me to do something I enjoy and I'm going to take advantage of that. But like Zed said, if you're not being cut in on the profits you owe them nothing but a day's work.

I write about myself all the time though. That's not what this blog post is about. This is a story about loyalty, commitment and questioning your devotion to anything other than what will bring you, only you and no one else, the most good.

I was at a family dinner recently and a distant cousin I hadn't seen in years was there. She was talking about how much she felt she was taken for granted at her current job and spoke like there was nothing she could do about it. She felt this way not because she didn't have any other options, she does. She doesn't have a great degree but she does have professional training in the field she wants to be in (horticulture) so she can definitely get a job elsewhere. What she does now is basically water plants in a greenhouse and sell them to walk-in clients. What she wants to do is landscaping and landscape arrangements. She's told her boss this but he doesn't care and always does it himself leaving her to the menial work. She also feels she's underpaid. The reason why she hasn't left is because she "gave her word" that she'd work for him this year, that "he could count on her". She doesn't want to go back on her word but he doesn't care about teaching her anything and hasn't given her a raise in the (I think) almost two years she's worked there.

Fuck him.

We basically spent most of the evening convincing her she doesn't owe him anything, that she should be applying elsewhere to do what she wants to do and once she's found something to give her two weeks notice and be done with it. She was still iffy about it.

Another story comes from my mom. We were talking about work on our way home and her work is absolutely terrible. Her boss is misery incarnate and the conditions are absurd. Their internet is ridiculously locked down and they can't stream internet radio even though everyone *has* to have headphones on because they're in a big open space along with sales people and any number of people from other departments that talk loud on the phone all day or are otherwise distracting. She processes insurance requests and claims (If you've asked for a quote on a life insurance policy in the last couple of months my mom might have done the paperwork. She hasn't been back in underwriting for long.) and basically spends he day thinking. The distractions are impossible to deal with without some sort of audio block.

But that's only the tip of the iceberg. They have mandatory overtime but are only allowed to mark 7.5hrs a day down on their time sheets. That one enraged me. If a company EVER tries to spring that on me there's absolutely no way I'll comply. The day someone makes me fill time sheets I'll fill them in honestly. No way they're going to get me to lie for their (dubious) benefit. Her manager thinks it makes her look good to get her team to do so much work in the "regular work week". Once one of her co-workers was going in for a painful operation and had the decency of telling her boss in advance so that she could plan out the work for the day in her absence. Bad idea. Even her co-workers told her she shouldn't have done that, that she should've called in sick the day of. Why? Because the boss gave her a laptop to bring home so she could do her work anyway while she was recovering. That's inhumane.

I hear these stories coming in from everywhere all the time. It's insane. It makes no sense. In my grandparents' day it used to be that you did your best for the company until your retired and they took care of you. Now the company tries to squeeze everything they possibly can out of you until you break down or have nothing left at which time they toss you aside and squeeze a fresh new employee.

Ridiculous.

It's this kind of crap that started the small to medium size company explosion. People like me watch this going on around them absolutely bewildered. How can you treat other people this way? How can you cheat, lie and destroy and still have people that are loyal to you? How can people be so masochistic?

So we opt out. Like I said before, I'm one of the few lucky ones. I have a skill set in high demand right now and I have a pretty much constant stream of job offers even when I'm not looking. I'm not going to work for BigCo. I wouldn't fit in. Instead, I get to be a prima donna and walk into work wearing board shorts and flip flops carrying my roller blades over my shoulder while others wear khakis or suits. I get to do interesting work and don't need to worry about trying to be promoted. Promotion in most places would mean taking me away from I like to do (design and write code) and instead force me to do what I want to avoid at all costs (dealing with stupid bureaucracy, office politics and paperwork). So I get to ignore crap like this and instead turn all of it on its head. If the high salary and corner offices are given to those that are well dressed, not the best workers, do I really want to be there? If promotions are based on who has cleavage and who shaves their legs do I really want to be there? Office politics are retarded and best left to the sort of people who thrive on celebrity gossip. I'll stay well clear of all that and work at a small company with a small team on an interesting project. I won't need to "dress for the job I want" because the job I want doesn't have a dress code. The job I want I wear board shorts and flip flops to.

Jeff Atwood was one of the talks I was really looking forward to and he delivered. His talk was about writing and whether writing was arguably more important than coding. The case in point he had was to make us think of any famous programmer we knew and how we knew them. Did we know them because we'd read their code or their prose? It's pretty clear that for nearly everyone the latter was the case. His conclusion was that no matter how much code you write, no matter how awesome it is, it never happened if no one saw it. And the best way to get people to see it is to write about. He encouraged us all to write a blog, if only for writing practice.

His talk was very edutaining and had a very Kathy Sierra style to it. Very little text (usually only a quote from someone) and mostly just one big image per slide. It works extremely well. You convey a message with your image and then the focus is back on you. Your audience isn't constantly pulled between what you're saying and the summary (or worse, copy) of what you're saying on the slides so it's a lot easier to stay in sync with the presentation.

By the time Jeff was done I was dead tired. I headed home for a shower before heading out with CUSEC people again. We went to a club and I decided to have nothing but girly drinks (between shots of whiskey). I got really disappointed when the bartender didn't know how to make an appletini. I did manage to try a martini and a couple of other 'tinis before I left kinda early because I could barely stay awake.

I can't wait until 2009.

For more CUSEC 2008 related reading here's some blog posts I found:

Zed Shaw

Tim Bray

Jon Udell

John Kopanas

Louis-Philippe Huberdeau

Skrud

Bernd Haug

Matthew Gallant

Anonymous

If you wrote a post about CUSEC I'd love to read it and link to you here. Just leave a comment or send me an email at smokinn@gmail.com

Even working on a combined total of maybe 8 hours sleep for the past two nights (only about 3 hours the past night), day 3 definitely kept me awake.

It started off with a tutorial on Factor by Zed. I'd already been putting around a bit with it but the keyboard shortcuts he showed alone made it worth attending.

Next up was a very interesting keynote talk by Jon Udell. Jon spoke of the implications of the social web, where we are now, where we want to be later and a little on getting there. The example he gave that most stood out in my mind (that represented where we were) was of a widget he wrote for librarians so that people could easily check before buying a book if it was available at their local library. To use it, librarians would have to match up the ISBN code from the url of whatever software they used to keep track of their inventory in some pattern that would allow his program to check.

And librarians could do this. They would look at the long cryptic urls, spot out the key information, find the pattern and then, once one person did it, Jon's software worked for everyone that used the same program for inventory.

The problem came when some software just didn't play nice. For example, some required a post. While it's definitely possible to decrypt what's being sent through post (firebug will tell you for example), it's not something we can expect non-programmers to try and figure out.

So URLs matter. They convey information to the surfer about where they are and what they're accessing. Had the software been configured to work through URLs like http://www.library.com/books/[ISBN # here], it would have been extremely easy to parse.

So REST is part of the solution. Tim Bray agrees. REST stands for Representation State Transfer. If you've ever built a rails app you probably saw it in action. If more URLs were built to be representative of the data they convey, interaction between websites would be so much easier. Even my blog is guilty of not playing nice. Why is this url http://smokinn.tengun.net/wordpress/?p=234 ? That's terrible. Not only does p= say nothing, my blog is bound to the software used to run it! Even something like http://smokinn.tengun.net/blog/posts/2008-01-23/1/ would be better. The first blog post on the 23rd of january 2008. Or maybe something based off the title would be better, simpler to remember. Either way, URLs say something to the reader and ignoring them is a mistake.

Jon definitely convinced me of this and a web framework that I planned of reworking and hopefully releasing as open source soon might need some serious refactoring. I want it to play nicer with the URL. Right now navigation is based on the "action" and everything is passed through index.php. So you end up with a url like www.site.com/?action=something&other=something_else etc. We already use an xml file for breadcrumb navigation (to tell us that something is a child of home) so I could use that for REST architecture as well. I'm just not yet sure how I can make the server ignore all the forward slashes sometimes and not others. I'll look into that soon.

Where we want to be he illustrated through an old Apple video (commercial?) where the guy seamlessly did research by talking to his computer and issuing very vague commands (Show me that paper a few years back about rainforest waterfall. I found this paper by this guy, that paper by that guy, this paper by that guy. Summarize them. Overall they conclude this but this other guy contradicts them saying that. etc) and the computer can get the data he needs to do his work. Now it's possible to get the same data but only if you have a day to fish around looking for what you want. Even then you'll probably miss a bunch of stuff. While Google's motto of making the world's information accessible is a great one, we need to make it accessible to everyone, not just the people who understand how to use search engines efficiently.

So, in the end, Jon was one of the three speakers to have the most profound lasting effect on me once CUSEC was over. One of the others I'll be writing about tomorrow, the last keynote, Jeff Atwood.

CUSEC Day 1 ended with a blast. Everyone headed out to the pub and, as far as I could tell, had a great time. I know I did. One of the highlights though were teaching Jeff Atwood and his wife to play bunnies. They are now officially apprentices, hopefully on their way to spreading word of the league through the world.

The other highlight was this amazing coding horror sticker he gave me.

He signed it for me too!

Isn't that so cool?

His keychain mastery came in handy as well when I dropped it under the table and found it thanks to his flashlight.

I only got home at 4 am but it wasn't that big of a deal since I knew that Julian had canceled his morning talk and the other didn't interest me all that much. I slept in (well, if you can call 5 hours of sleep sleeping in) and showed up for Dr. Grogono's presentation.

Dr. Grogono spoke about Erasmus, a language he is writing to help deal with concurrency issues. Trying to keep it simple, the language seems to be based around 2 concepts only: cells and protocols.

If I understood right, cells are a little like classes, in that they encapsulate their own state and behaviour. They also define protocols for communication, stating what they expect to receive (types) and what they'll provide in return. Other cells implement these protocols and this allows for message passing between cells. Since cells are independent from each other, the generated code can be executed in parallel. Dr. Grogono said he already has a prototype compiler (it compiles Erasmus code to C++) but it needs to be rewritten. I'm going to keep a lookout for when it's available and when it is I'll probably try playing around with it a little. It seems pretty inspired by Erlang (the cells being kind of like processes and having nothing but messages being passed between each other) and could probably work just as well while being simpler to use.

The other talk of the day was by Dr. Ullman and, honestly, I didn't get much out of it. By then I was dead tired and slept through most of it. I noticed he talked about minhashing and Google's pagerank but that's all I remember.

The last talk of the day that I attended was a presentation on building a software startup by Idee. The starting a startup talk was like many I'd heard before, what was interesting about this talk was the software that they develop. They showed a demo of their image recognition software and it was just plain amazing. They have the demo available on their website and I highly recommend you check it out. I haven't been that impressed since I saw the video of smart image resizing algorithm.

At the end of the day we had the banquet which was ridiculously long (from the time I sat at my table around 7 it took over 2 hours for my meal to show up) and when my steak finally showed up it was raw. Not rare, raw. About 2cm of meat of the outside was seared but it was thick and the inside wasn't cooked at all. Since I like my steaks well done I didn't even finish it. Oh well, time to go party again.

Or not.

There was nothing officially planned for saturday night which might have been a mistake. People just kind of scattered all over. Since Skrud is ridiculously popular, when we said we were planning on going to distillerie a lot of people showed up there. It's already a fairly popular place and it's not very big so by the time I showed up it was so packed they weren't letting anyone else in. We went to another bar and too many people showed up there so we tried moving again. After we couldn't fit everyone into a third bar we gave up and split up. Skrud, Alex, Harley and I went back to Skrud's place where we found out that Jackie was at Brutopia so we headed for a pint there. By the time we got to sleep we had to be up a little over 3 hours later for Zed's factor tutorial the next day. The talks of day 3 had better be engaging or there was no way any of us would be staying awake....

Zed Shaw was nothing short of brilliant. His talk was funny and engaging and also very useful, exactly what you're hoping for every time someone walks up to the stage. To give you an idea of the kind of guy Zed is, he wrote his slides in Factor, a stack based language. His Factor code compiled down to TeX which was run through OpenGL to show the slides. So awesome.

Correction: The presentation was written in Factor using a "TeX-like" DSL that mapped to an OpenGL GUI provided by Factor.

If I had to sum up Zed's talk in a single sentence, it would be: How to work for a big corporation without losing your soul. He started by talked about ACL lists and how terrible they are, stemming from the simple fact that they aren't turing complete. His team re-implemented a massive, near-unmaintainable ACL system consisting of something like (I hope I'm remembering this right) 270 000 ACL business rules with about 4000 lines of Ruby. And not only was the Ruby solution smaller, the non-programming employees could actually read the rules written in Ruby directly and say yeah that's right or no that's not!

The system they had was slow, it wasn't flexible and it couldn't even do some things (because of the lack of turing completeness). You'd think with so many major flaws on one hand and a good, working solution on the other (the Ruby program was already written and worked), the better solution would be picked every time, no?

Of course not. Anyone that's worked for a big company or even just read dilbert knows that efficiency and smart decisions are the least of corporate america's motivations. What motivates business decisions? Steak and strippers.

Zed argued that the average business oriented manager (someone in a company that does not produce software as a product, a company whose money is elsewhere but needs software internally) doesn't understand programmers, doesn't understand their motivations and because of that they fear and despise them. They wish they could get by without them but they can't. They need the software even if they understand nothing about it. They much prefer someone who's trying to actually sell them something because that they understand. Plus, free steak and strippers.

So what can we do about this? Zed gave us many solutions.

First of all, we must get more techies into management positions where they can make a difference. A lot of techies now do this by getting an MBA. DON'T DO THAT! The first step of an MBA is lobotomization. You go in carefully weighing the benefits and options of two positions based on facts and you come out spouting crap like Synergy! We need the software that will create the most Synergy!

We also need to be brutally honest when dealing with managers. This means exposing both the pros AND the cons of any solution. We have to stop trying to be salesmen because we're no good at it. We'll always get beat by the career salesman. You have to short-circuit this sales idea and make it about a real decision, not about sales.

Finally, we need to stop fetishizing the Technology Of The Week. Rails is the best! Erlang solves everything! No. We need real practical decisions based on real value. Something we know will work and not something just because it's cool (to us).

One of the most important take-aways from his talk was the fact that you don't owe the corporation anything more than an honest day's work. Too many people are so happy to simply be employed that they give everything they have to the company and basically allow themselves to be utterly exploited. Unless you have some sort of profit sharing agreement where you actually get some of that extra profit that your extra work generated, you shouldn't be slaving away giving all your best ideas and work to the company.

Instead, be creative. Do work stuff at work and fun stuff at home. Crazy stuff that has nothing to do with anything, stuff that's not the least bit useful, stuff that simply fun. Not only does it make you a better programmer, it's refreshing and just plain fun. The more pointless, the more crazy, the more absurd and useless, the better.

That last idea actually got me thinking about work situations a lot. I've started a little draft on that idea but how I want to convey it isn't entirely crystallized in my mind so I'm going to wait until I'm done my CUSEC series before publishing it. Just the fact that Zed got so many people reflecting on their life, their situation and what you can do about it made the talk a success. What made it amazing was Zed.

I know I couldn't possibly have done it justice with this post. The CUSEC organizers said that they taped everything in HD and promised to make everything available to download so I urge you to watch Zed's video when it comes out. I'll make sure I make another blog post when the videos come out.

To give you a representative idea of the impact Zed made on everyone present, check out this feedback form the organizers gave everyone at the end of the conference. This wasn't even my form either. (The quality isn't great since the pic was taken with a camera phone.)

Yeah. It was that good. There were actually multiple forms that had "Zed Shaw" in every answer.

Zed Shaw won CUSEC.

CUSEC started with a bang. Right off the bat we got a talk by Tim Bray, an inventor of XML and Atom who was CEO of Waterloo Maple for a while and now works for Sun. He talked about what we as a software engineering/computer science community do well and what we dont.

First, what we do well. We've managed to create a seamless platform that runs on any computer running any os with a very low barrier to entry. So if you want to do something fun and interesting, that's a good avenue to follow. More on this platform later.

There are some things however that we do extremely badly. Two of the worst are Integration and Concurrency.

When talking about integration, he mentioned how terrible we are at making programs communicate between each other. This was actually a very pervalent theme among keynotes this year since Dr. Grogono and Jon Udell also spoke about the communication and interoperability problem. Unlink most however, Tim Bray offers at least an avenue for a solution. We tried Corba and it failed. It's extremely hard to get working and once you do it's not at all flexible and very very brittle. We tried SOA and the jury is still out on that. Some people seem to get it working and some others fail miserably. So perhaps SOA holds at least some elements to a solution. The avenue that he claims has most promise however is the one platform that runs on all computers seemingly efforlessly, that everyone knows how to use and has been working well for a very long time. This platform isn't a virtual machine. While interesting small scale solutions to share between applications, you're stuck with a single stack. The JVM lets Ruby programs communicate to Java programs but not C# applications since they run on the .NET (or mono) VM. The platform in question is actually the internet. If we structure our programs to conform to the basic tenets of the internet, we can get them working together. He mentions REST from Roy Fielding's thesis as a useful structure for exposing your programs to others for access. I was going to write a little about REST but since a large part of Jon Udell's talk was based on REST style architecture and the social implications of it I'll leave that until then. For now, if you don't know what REST is you can check out the Wikipedia article on it.

So while a RESTful architecture for new programs would be a great help for the future, finding a solution that would let RESTful programs communication with legacy applications is a problem that still needs to be solved.

Another huge problem is concurrency. As Tim says, Moore's Law has gone sideways. We ran into some fundemental laws of physics and can't really increase clock speeds that much. At least not until we realize there's a way to go faster than light!

Image Source

As that chart shows, Moore's law is still holding. We still have more transistors. But the effects are no longer the same. Instead of increasing speeds we have an increase in the number of cores. While this is theoretically interesting, it won't be truly practical outside a few domains such as video encoding until the people who write software get much better at exploiting it. Tim mentioned the experiment he's been running to see how much a theoretically IO bound program can be sped up by exploiting more cores. The results are surprising. From the initial simple Ruby script that ran in 1 hour 43 minutes, the run time is now down to 1 minute and 51 seconds, quite surprisingly a Perl script.

But we still have a problem. The Ruby program is 13 lines of code. The Erlang programs clock in at over 300 and the Perl solutions, while smaller are still something like 5 times the amount of code. We know that more lines of code means more bugs. And not only is it more code, it's vastly more complex code. It's a breeding ground for problems. Which is why we have libraries to help us and language features for simple threading.

Again however, while we do need better solutions for writing future programs with massive concurrency in mind, we also need to take into account older programs. The Ruby program actually ran slower on the massive 8 core server than it did on Tim's laptop. If someone could find a way to take existing code and parallelize it automatically, that would be an incredibly huge advancement. But it's hard. Very hard.

Lots of very smart people have been butting their heads against these problems for quite a while so it isn't a space for the faint of heart. But with enough new ideas and enough work, we just might be able to solve them for good.

A small aside: Tim provided my only big dissapointment at this year's CUSEC. It wasn't his fault at all, it was entirely mine. When I saw the excellent Beautiful Code book on one of his slides I nearly kicked myself. I'd forgotten to bring it! I'd wanted him to sign it for me. I brought it the next day but since Sun bought Mysql, Tim couldn't stay. Hopefully one day I'll see him again. I would certainly enjoy attending another one of his talks.

Day 1 was absolutely amazing. Somehow, I really don't know how, every year CUSEC outdoes itself each and every year I go to it.

This year is the first I don't go as a student. This year I'm a "corporate attendee". All that really means is that I have to stand around my company's booth during the "career fair" hour of the day. On that note, the career fair got a lot better this year. Two years ago CUSEC had 3 companies present. Last year maybe 6 or 7. This year there's probably a dozen or more. So that's great for people looking for jobs.

This year I'm going to try a different format. Last year I wrote a separate post for each presentation I attended. This year I'll try to sum up a day in a post. Hopefully it won't ramble too much. If I find it does I might split it up later.

So, Day 1. Day 1 saw Tim Bray and Zed Shaw as keynotes. Both were amazing but more about them later. First, I'd like to talk about the academic presentation given by Marsha Chechik. Her presentation was about software verification. Software verification is something that I would never see myself investing time in because personally, it seems like very little return on investment for me. I'm selfish. But I'm really happy that some people invest time in it because it shows amazing promise for the future of software. Now I honestly don't see any major advances from this field but the incremental advances are quite obvious so I imagine that by 2020 you'll easily be able to go home and, while sleeping, have an automatically generated test suite run over your program on one box and have a verifier like the one Dr. Chechik is building on another.

From what I understood, the software she's building basically abstracts out your program to a basic skeleton of itself and tries to reason on it. If the skeleton doesn't have enough information to decide anything or seems fine, they then add more information again and run the process again, looking for errors. As Dijkstra said (and Dr. Chechik quoted): "Program testing can be a very effective way to show the presence of bugs, but is hopelessly inadequate for showing their absence." So while these techniques will never absolutely produce perfect software, I believe they'll get us a whole lot closer.

I realize now that my idea of summarizing an entire day in a single post was misguided. Way tltr (too long to read). So coming soon: Tim Bray speaking about hard problems in Computer Science and Zed Shaw being so fucking awesome.

If I told you my code was running slow, what would you do? Too many people look at the source and start "optimizing".

They see:

for(int i = 0; i < array.length(); i++)

{

// do stuff

}

And say aha! Everyone knows you should be storing the value of the array length, not re-evaluating it unnecessarily!

The code gets refactored to

arraySize = array.length();

for(int i = 0; i < arraySize; i++)

{

// do stuff

} 

This should help, right? It won't. Or not nearly as much as you think it will. The only way to speed code up is to profile it and remove the largest bottleneck. It's the only way. You'll never meaningfully speed up a program by guessing. You'll always be wrong. If you accept that statement you'll waste much less time optimizing uselessly. Saving a few milliseconds total on a loop is worth nothing if the database or RPC call inside is taking 2 seconds.

Performance Myths abound and performance myths are only a small part of what I wanted to talk about. The title holds true for much more than just performance metrics.

What initially prompted this post was a very brief conversation I had while reading the excellent Best Kept Secrets of Peer Code Review book. I was reading a passage out loud that had surprised me:

From Figure 24 it is clear that review size does not affect the defect rate. [Ed: the defect rate is the number of defects found per hour] Although the smaller reviews afforded a few especially high rates, 94% of all reviews had a defect rate under 20 defects per hour regardless of review size.

After reading this to her she said really? Doesn't seem like that would be the case. I thought the same. Both our intuitions was completely wrong in this case. Whether you're reviewing 100 lines of code or 400 lines of code, the number of defects you'll find in a hour is pretty much the same. Amazing.

This is why you should never, ever trust your intuition. You can't afford to. It might be right sometimes but very often it will be dead wrong. The only way to make a proper decision is to base it on facts and measurements. Perhaps these will be in line with your intuition, perhaps not. But if they are, the time spent measuring was not lost. The time spent measuring when intuition was right is the small payment you make for all the times you measure and see your intuition was completely wrong.

This particular topic is very important to me because I came very close to making a colossal mistake. PhpEd essentially saved my project back in July or August or so. When I graduated, my first task as a recent grad was "new everything". The company wanted me to redesign and reimplement all their web based software. Some of the stuff they were doing just wasn't maintainable anymore and they wanted a fresh start. A couple of others and I took on the task of designing and implementing a PHP MVC framework from scratch. We wrote our own ORM layer, had some form validation tools and some other neat stuff. When we had written our first web application with the new framework, page loads were ridiculously slow. (As in 10, 15, 20, sometimes even 30 or more seconds per page load.) We saw that the slowest pages were the pages that made heaviest use of objects so our intuition told us we were creating way too many objects and using too much memory. Our intuition was wrong. I ran the phpEd profiler over our code and noticed that something like 90 or 95% of the processing time was spent in __autoload(), a function that automatically loads a class should it not already be defined. (So you don't have to sprinkle file includes all over your source.) We changed it from a naive recursive directory search to an in-memory hash of file paths and we instantly got sub-second page loads. We were about ready to change our entire core architecture and revert to php4 style code just because of a hunch. Measurement saved me.

So, please, never trust your intuition. If you have a problem, never base a solution on a guess. Because that's all intuition really is. And like the code review book shows, this can and should be applied to much more than source code. Your process can and should be measured. Vague guesswork and tradition will not get you nearly as far as figuring what truly works and applying it.

I just read Larry O'Brien's latest post titled No Silver Programmers and must say I completely agree. I agree but I also have a little to add.

I think the problem I (and Larry, though I don't want to put words in his mouth) have with the quote "5% of programmers are 20x more productive than the other 95%" is the absolute grouping it seems to give people. It seems to say that either you're 20x more productive than everyone else or you're a loser who's outclassed by an elite 5%. As is obvious (or should be) to anyone who knows or has met even a dozen humans, humans aren't easily categorized. Nature however seems to be fairly predictable and while humans can't be easily categorized, we are generally pretty easy to plot.

The normal distribution, better known as the bell curve. It's what I generally resort to thinking about when others start talking in absolutes. I have no problem believing that someone in that 20+ area of the curve is 20x more productive than the guy in the 5- area. But then again, that 5- area is why we end up with FizzBuzz. 20x is even being generous to the guys at the bottom. I'd have no problem believing infinity times since the top tier can do stuff the bottom tier won't even be able (or, more accurately, willing) to figure out, let alone write of their own volition.

(In the following I use the masculine tense just because it's easier. A top programmer could just as easily be a woman.)

I don't even have trouble believing that a top tier programmer left alone with a project could finish in a day what a middle tier programmer left alone would finish in a month (although he would have to already be an expert in the domain and the domain would have to be hard, not common business software or web development). I could name names of people who would easily finish stuff way faster than I possibly could. However, I don't believe that at all when it comes to teams. A good team is a lot more than the sum of its parts. Lets say you're shown two top programmers and given 4 average programmers. You can only pick one of the top programmers. One works well with a team but the other likes to lock himself in an office with a week's worth of food and only come out when he's running out and all the perishables went bad.

Disclaimer: the following "math" is based completely on intuition and even only vaguely on personal experience. It is most definitely wrong and I sorely wish there were some real studies with real data.

Let's say the tent dweller forms team 1. The average guy gives you 1x productivity and your top guy gives you 20. Each works in his own little world and you have 24x productivity. (Haha, yeah right.)

Now let's have a look at team 2. Your top guy has to sacrifice time to explain what he's doing, why he's doing it and how others can do the same to the team. Let's say this takes a huge toll on his productivity and he can only get half what he'd normally do done. (Although it's often the opposite, by explaining and bouncing ideas off other people you often end up with better ideas and spot obvious holes but let's assume for now he's perfect.) The time he spends though helps bolster the others on the team. If everyone is up to 5 and your top programmer is down to 10 you win since you're now up to 30x productivity. (Although probably much more since you avoid all the integration issues of having the whole team work as a whole as opposed to individually.)

So while I do believe that some programmers may be better, I believe that it's much more important to have a team that works well together and that a team that works well together will beat a team of great programmers that don't. Of course a good team doesn't guarantee success, but a bad team just might guarantee failure.

Considering there's less than a week left in this one I'm pretty sure this is my movie of the year.

My movie of the year is actually from 2002. And, surprisingly, it's a tv movie. It's called Bang Bang You're Dead. I considered writing a short review/plot synopsis but I won't. All I'm going to say is watch it. It's great.