The code etymologist

There is a general belief that spoken languages emerged more than 100,000 years ago. Even though the first writings were invented around 3000 BC, formal education was introduced much later, in the Middle Ages. However, it was limited to churches, monasteries, and the upper class.

Mass public education started around the 17th century. So, you can imagine that all this time, most of the communication was in spoken form. Each retelling of the same story distorted part of the details, little by little. So, it’s not surprising that today we’re unable to distinguish between myth and fact.

Even words themselves and their meaning changed over time. We know that thanks to etymology, which studies and documents the origin of words.

Today, we live in a world with real-time worldwide video communication, collaborative drawing tools, virtual reality, LLMs capable of reading text, creating poems, and generating videos. In contrast, software development teams often use ancient methods to pass knowledge from one member to another: verbal communication.

As a new team member, the onboarding process becomes an adventure. We must search for the knowledge we’re looking for. Sometimes that knowledge was distorted in time. Different team members know different information. Some members departed, leaving a gap in the knowledge which must be rediscovered. Involuntarily, we become code etymologists.

"There is no documentation!"

When was the last time you heard, or even said it yourself: “There is no documentation”? I have started hearing and saying it myself once I began working on large projects, which spread over multiple years and had multiple team members.

So, we have to ask ourselves, "Why is there no documentation?"
The obvious answer is that "Because nobody wrote it."

But, "Why hasn't anybody written it by now?"
Well, "Because it takes time and effort."

If it works without it, why bother, right? However, the cost will eventually be paid, at a later time, with interest. Team members, new or old, will eventually search for answers. They will have to find the right person, which will have to recall the details or even rediscover lost knowledge. This too requires time and effort. The difference is that this scenario will occur multiple times.

The problem is that it's too difficult to measure the cost of missing documentation, in contrast to measuring the time it takes to write it.

Agile Development

One possible explanation for missing documentation is the practice of agile software development. Its primary values are defined on the homepage of the Manifesto for Agile Software Development, and the first two of them are:

1. ⁠Individuals and interactions over processes and tools
2. ⁠⁠Working software over comprehensive documentation

Reading the above can easily be misunderstood as "we shouldn't write documentation, instead we should talk and interact with each other". Humans often see the world in black and white. It's simpler and easier to reason about. Therefore, we tend to oversimplify complex information.

That's a logical fallacy. Even the manifesto explicitly states, "That is, while there is value in the items on the right, we value the items on the left more". In other words, preferring apples over cherries doesn't imply you hate cherries and never eat them.

However, the 6th principle says that "The most efficient and effective method of conveying information to and within a development team is face-to-face conversation." Now, that could be a strong argument for someone to believe that the only communication method is verbal face-to-face conversations.

I strongly agree that the most efficient way to communicate is in-person meetings. But when this becomes the only way to communicate, two problems emerge:

1. We end up wasting time repeating the same information over and over. According to Metcalfe's law applied to social interactions, peer-to-peer communication effectiveness is inversely proportional to the number of team members. In other words, the larger the team, the less effective and efficient verbal communication becomes.
2. Humans have selective memory, which leads to confirmation bias. As time passes, it becomes debatable which information is correct and complete, since different team members will recall different details.

In the end, the Manifesto for Agile Software Development doesn't suggest anywhere to have no documentation at all. There's even a hidden History page which clearly states that:

We embrace documentation, but not hundreds of pages of never-maintained and rarely-used tomes.

Authors

The next question we should probably ask is "Who's responsible for writing documentation?" Good question. Who should write it?

Could it be the "documentarists"? Well, that's a term mostly used in the film industry. They are responsible for creating documentaries.

What about "technical writers"? Well, they are usually focused on external user documentation. They write manuals, guides, and tutorials. This article focuses on internal documentation, which is aimed at the team members.

So, there is no clear role responsible for writing internal documentation. It is just one of the various skills that software developers should have. And these skills overlap to a large extent with the responsibilities of a dedicated technical writer.

Forms of documentation

Software development is a complex process, therefore, its corresponding documentation cannot be simple. Depending on what exactly we want to document, there are various forms to record and communicate that knowledge:

• Text is the most basic and simple form to record and share knowledge. Written documentation equally applies to relatively small content, such as code comments, project setup, and to larger content, such as coding guidelines, technical decisions, requirements specifications, etc.
• Graphics provide a more intuitive alternative to text, which helps us visualise information in a concise manner. Graphical approaches are best suited for documenting development workflows, or explaining how a system works using flow charts or sequence diagrams.
• Code has the added benefit of being executable, besides representing a source of documentation as a secondary trait. Documentation as code is suitable for automated tests, data structures and contracts, deprecation warnings, or defining infrastructure using Dockerfiles.
• Tools are great at enforcing practices for all team members to ensure a consistent development process. Such practices include code formatting, linting, or constraining a specific Node.js version.
• Interactive interfaces provide a whole new level of documentation, which is helpful both for visual elements such as UI components or design systems, but also for non-visual systems such as state charts.

In closing

Documentation, in a general sense, is about communicating knowledge, which can be achieved in different ways. We can speak. We can write. We can draw. We can code. We can enforce certain activities. We can even build interactive experiences.

If we don't get passed the speaking method, without any doubt someone will eventually become a code etymologist, doomed to search for lost or opaque information.

We all appreciate good documentation.
We all blame the lack of documentation.
But still, we avoid writing it.

In the end, code etymology is about reading the source code, refactoring it, talking to team members, asking questions, understanding decisions, and eventually documenting them so others won't have to go through the same effort.

Since "missing documentation" usually has different meanings, depending on who or when we're asking, in the next chapters we'll explore several areas of software development that often lacks documentation.

Continue reading Part 1: Project setup