“Overengineering” is a word that gets thrown around a lot. It’s used in a negative connotation, but I have a hard time defining it.

It’s not the same as Premature Optimization. That’s when you add complexity in order to improve performance, at the expense of readability, but the payoff isn’t worth the cost.

If “to engineer” is synonymous with “to design”, then overengineering is spending too much time designing, and not enough time… what? Implementing?

Lets say you and I need to travel across the continent. You despise overengineering, so you set off on foot immediately, gaining a head start. I go and buy some rollerblades, and easily pass you before the end of the day. Seeing me whiz past, you head to the nearest sporting goods store and buy a ten-speed. You overtake me not long after breakfast on the second day. “Hmm,” I think. I don’t have much money, but I rollerblade on over to a junk yard and get an old beater car. It doesn’t run though. I do some trouble-shooting… the electrical system is fine, and we have spark, but we’re just not getting ignition. I might be able to fix it, and I might not. Should I go and buy a faster bike than yours, and try to catch up, or should I take my chance and see if I can fix this car? I’m pretty sure I can fix it, and if I can, I can easily win, but if I can’t, I’m giving up the lower-risk but lower probability chance of winning by getting a bike.

It’s this last type of choice that we’re faced with as engineers. You have a project with an 8-week timespan. We estimated that it will take 10 weeks at 50 hours per week using standard practices, so the project manager just wants everyone to work 60+ hour weeks using the same practices because from their point of view, that’s the “safe” bet. As an engineer, you might be able to spend 3 weeks building something with a 90% chance of making you 2 times more efficient at building this type of project: 3 weeks spent building the tool, and then it would only take 5 weeks to complete the project, so you’re done in 8 weeks. Not only that, but then you’ve got a tool you can re-use the next time.

If every time we had this choice, we chose to make the tool first, then on average we’ll end up much further ahead. Every time we succeed (90% of the time), we’ll greatly improve our capabilities. We’ll out-innovate the competition, with lower costs and faster time to market. However, a manager is much more likely not to build the tool because they can’t tolerate the risk. The larger the company, the worse this is, and the typical excuse leveled at the “tool” solution is that it’s “overengineering.”

Imagine we’re back in the cross-continent scenario, and I’ve decided to fix the car. Two days later I’ve got car parts all over the ground, and I haven’t moved an inch. Meanwhile, you’re a hundred miles away from me on your bike. Who’s winning the race? You can clearly point to your progress… it’s progress that anyone can clearly see. I, on the other hand, can only show someone a car that’s seemingly in worse shape than it started in, plus my inability to move over the last few days. The pressure starts to mount. It’s surprising how quickly people will start to criticize the solution they don’t understand. They’ll call me a fool for even attempting it, and applaud you on your straightforward approach.

Of course you realize that if I can get the car working, the game’s over. By the time you see me pass you, it’ll be too late to pull the same trick yourself. I’ll already be travelling fast enough that you can’t catch me. If there’s a 90% chance I can fix the car, I’d say that’s the logical choice.

So is fixing the car “overengineering”? If the race was from here to the corner, then yes, I’d say so. The effort needs to be matched to the payback. Even if the race were from here to the next town, it wouldn’t give you a payback. But what if we were going to race from here to the next town once every day for the rest of the year? Wouldn’t it make sense to spend the first week getting myself a car, and then win the next 51 weeks of races?

In business, we’re in it for the long haul. It makes sense to spend time making ourselves more efficient. Why, then, do so many companies have systems that encourage drastic short term decision making at the expense of long term efficiencies and profit? How do we allow for the reasonable risk of failure in order to collect the substantial reward of innovation?

You start by finding people who can see the inefficiencies — the ones who can see what could easily be improved, streamlined, and automated. Then you need to take those people out of the environment where every minute they’re being pushed for another inch of progress. Accept that failure is a possible outcome once in a while. Yes, there’s risk, but there are also rewards. One doesn’t come without the other.