Why do we write design documents?

Starting a greenfield project is loads of fun, but you can’t avoid the dentist’s appointment indefinitely and at some point you will have to write that design document your boss expects to see on their table. Despite the fact that most people learned how to write in childhood, more often than not they are struggling when it comes to writing lengthy documents. The big blank page is daunting and it’s hard to get into the mood for writing, because it’s time consuming and not as fun as programming.

How do I even start explaining what we are trying to build when I don’t have the slightest clue about the thing we are building? It’s just an API this middle manager put me in charge of, why would I waste my time with writing the design document when by the time I am finished writing I could already have the API up and running?

If the API is so simple that even a kid can understand its purpose, isn’t writing a few pages about it a trivial thing to do? I am sure you can solve a primary school’s math problem without breaking a sweat, so why would you struggle with filling the paper when it comes to describing the behavior of the API?

This writing block usually happens when you don’t know enough about the problem that you are trying to solve. If you worked on the large project for a prolonged period, your brain is most likely filled with details and problems surrounding that project. If you were to write them down you could probably fill thick books, nevertheless very few people are able to do that at the beginning of their project. Their minds are as blurry as is their API design.

Essayist are writing their essays, because the act of writing helps them to think and form their thoughts. They operate at the levels of thinking where their brain can no longer process those thoughts, so they have to write them down before they disappear in the cobwebs of their mind.

Everybody was staring into that direction, because they were standing on the so and so’s shoulders. Lo and behold, we stared in the wrong direction and the only reason why that was discovered was due to the ramblings of some essayist who was thinking through their writing. That doesn’t mean that everybody who is forming their thoughts through writing is worth reading about. Lots of writing finds its best use as a toilet paper, so if reading these words induce some bowel movements you know what your next steps should be.

I mostly see two reasons why larger projects demand writing a design document and other kinds of paperwork bureaucracy. The first reason has educational purposes and is there to produce some kind of guidance documentation that the rest of the team can rely on during the actual development. It answers, or rather it should answer common questions and provide general guidance on how to go about solving the problem.

Less discussed part of the secrets behind writing the design document is to solve the same problem that the essayists discovered hundreds of years ago. Sometimes the problems are so hard that they simply can’t be solved only within your brain and you have to expand your brain power by offloading the thinking process to paper.

When you start designing a new project, your mind is usually a spaghetti mess of questions. What is the problem? Which technology should we use? When was the last time I ate? What do the requirements say? How are we supposed to deliver such and such performance? Oh boy, this team is going to be pissed when I tell them to redesign their service.

The act of writing lets you discover problems that you didn’t think about before, as you simply can’t fill 20 pages of coherent writing about the problem without having a clear picture in your mind ¹.

But, they are always outdated

The most common arguments that you may hear against writing the design documents is that they are not going to capture all the details or they will go out of date and become irrelevant. People won’t be reading them because they don’t have the time or they don’t care enough about the topic and yadda yadda.

While those arguments might be true, they entirely miss the point and are simply excuses for putting out a half baked undocumented project. Unless you rely on formal verification or heavy quality assurance process like they do for space projects, there will always be edge cases in your system that will only be found out in production. The systems that are being developed these days are far too complex for one person to envision all the possible edge cases beforehand.

Writing a design document before programming the complete implementation allows you to avoid a much larger problem - marching in the wrong direction based on false ideas. “Whoops that was dumb, let’s not do that anymore!” in a large team costs far more than taking a few days/weeks off in order to think and write down your thoughts beforehand.

Sure, some companies might take this into extremes and will demand a closet filled with documentation when it comes to building a shed, because according to their processes every project needs one. Well no, not every project needs one, but every project will surely benefit from some additional thoughts and documentation.

Amazon went one step further and banned all Powerpoint presentations in favor of writing lengthy full sentence documents. They argue that the Powerpoint cabbages ones mind, and the truth is, most Powerpoint acolytes that I’ve seen had their minds made out of cabbage. Powerpoint makes it easy for a presenter to just throw some bullet points on the screen and hand wave it through the sloppily made presentation. Trying to drill down more into the subject after such presentation is unlikely to return a satisfactory answer, because they didn’t think it through ².

The brain itself is a very lousy organ when it comes to retaining information and it’s easy to overwhelm it with the amount of information that a hard problem requires. When your memory runs low, swap your thoughts on paper.

Notes