Why Big Teams Don't Work

I read somewhere that Microsoft has a 24-hour system for software development with automated builds going on in the night and literally hundreds of programmers on their bigger projects.

The obvious question, then, is why isn't their software dozens of times better than everyone else's? It's not only "not as good" much of the time, or "marginally better" occasionally, but it's full of bugs, expensive, and, well, just not that great. Microsoft has more money than anyone else on the planet. They're doing what they think is best. They have not bought too many copies of The Mythical Man Month, or read them.

So what's on the other end of the spectrum? Probably shareware, which typically has a single author and is a labor of love. There's lots of really good shareware in the world, some of it, frankly, rivals Microsoft's offerings.

So why is it that small teams work better than large ones, or at least anecdotally appear to?

I have a lot of theories, and for the record, I am a proponent of small teams and have never run a software project with more than 5 engineers, including myself.

My principle theory is simply that projects involve human beings, and as a species we don't handle complexity very well. We are wired to simplify: vision simplifies what is really there, recognizing patterns; socially we simplify: you're either Good, or you're Bad, Guilty or Innocent; intellectually we simplify: the scientific method is based on reducing an experiment down to as few variables as possible so you can "control" for them and measure the one you're interested in. Humans can support up to 3 simultaneously contradictory thoughts at once, before melting down into indecision and confusion.

So what happens when you put 400 programmers on one project and try to run it?

The first thing that happens is you need at least 50 managers and project managers and coordinators and cross-functional liaisons and whatever else, just to try to "communicate" about the project. Right, like that's going to help.

The second thing that happens is nobody can really get work done any more. You try to edit a file, make some changes, add a new parameter to an API somewhere. You can't just check the file into CVS (or SourceSafe, in the Microsoft case). You have to find all the people who are using the API and let them know. You have to figure out if anybody else is modifying the same file, for different reasons. You have to stop working, and talk about working instead. God forbid it turns into 3 or 4 meetings, a few specifications, and, conservatively, about 50 email messages.

I am a programmer, among other things. I've changed API's many times. With a good search-and-replace tool and a thorough understanding of all the places an API is used, you can make a sweeping, and I mean sweeping change, without any errors, fully validated and tested, in a couple of hours.

So efficiency is part of it. Confusion, cross-currents, communication, and all the things that fall out from that: backlash, politics, sandbagging, employee turnover, morale problems, managers on a tirade trying to "fix" it; you may have seen some of this. It's preventable, believe it or not.

Just use small teams. Small teams are how the best work in the world gets done. Even if it's a "big project". How many people do you think it took to build the Golden Gate Bridge? I don't know, unfortunately, but I bet you it was at most a couple hundred. If Microsoft built the Golden Gate, do you think they'd put fewer than 2,000 people on it? Do you think it would be on schedule?

I don't mean to pick on Microsoft, though they're the biggest, most resource-heavy example of large-scale product development on Earth. They're certainly not unique when it comes to large-team molasses-style development.

Maybe this is why "outsourcing" is becoming popular. Hey, maybe if we abdicate this project completely and give it to a team in Russia (translation: 1 or 2 really smart people) it will come in on time and under budget?!

Here's a suggestion: outsource it to your 2 best engineers instead. Set them up in Marin with no phone lines. They'll get it done, I promise.