Documenting systems

Back when I was working at virtualQ, I had to do some documentation. At first I didn't quite enjoy it, feeling like this was a chore. But documenting a system can be a very interesting task. I also had to explain our systems to new hires after we raised some funds and quickly grew the team. 

We had a documentation website for our API, based on markdown. It was very technical, with manually copy pasted sample API responses. It wasn't very visual. It had endpoints documented publicly while they were only for internal use. It had a lot of outdated content, and felt more like a patchwork of data bits than a comprehensible thing.

My fist step was to remove the documentation off irrelevant endpoints. The docs were kind of an hybrid between a public api documentation and some internal guide for new hires. I made dedicated to customers trying to use our api only.

It still had a "wall of text" feeling though. To make it more palatable, I tried to add visuals. Integrating mermaid to the docs was quite easy, I styled it roughly with some css rules. Sequence diagrams are awesome. I used them to communicate the different workflow we had and some typical scenarios the clients would like to use our API for. 

We had a complex state machine that tracks the state of a caller through our system. I tried to represent it as a graph, but that was a bit too cluttered, so i split it in two graphs and removed some of the transitions that were not normally happening. 

Finally, I wanted to fix our outdated sample API requests and responses. Our API was evolving rapidly and having outdated samples was more confusing than helpful. I created a tiny script that curl commands and update the API answer samples with them. I had to do some dirty grepping on the answers to replace dates, ids, etc .. to avoid polluting the git log with meaningless edits every time we'd refresh the samples.

We deployed the new version of the documentation. Some of our new hires still had some use for the old version (that documented more endpoints) so we added a link to it. We don't have any tracking on the docs website, so I have no idea what the readership is, but it was great as a learning experience.

It made me love mermaidJS so much, that I eventually ended up building a google docs addon for it. 

I like the idea of writing tutorials as markdown, with E2E test instructions mixed in, and outputting the result (text, screenshots, response samples) as a document. It would guarantee that the tutorials stay in sync with the product and don't become outdated, with always fresh screenshots. It would make the task of writing E2E tests for some web app a dual purpose task, where you'd generate both a helpful document and some smoke tests in one go.