Docs Aren't Code

In the world of development, the need to track bug reports and enhancement requests are a given. But they're not generally required for documentation, in the way they are for code Quite the reverse. For documentation, bug reports and enhancement requests provide little benefit, and generally impede progress. (Reports filed by users are an exception, but only if that is the only means they have to provide feedback.)

This post compares documentation and code, showing why bug reports and enhancement requests are so vital to the code base, and at the same time why those reasons simply do not apply to documentation.

This table highlights the differences between code and documentation, to show why bug reports are basically superfluous for the latter:

Code Documentation
Code is opaque.
When a programmer changes N = A \* B + 1 to N = A \* (B + 1), the rationale is anything but clear. A bug report is needed to explain the intent of the change.
Documents are transparent.
When a writer changes "the system is intended to do X" to "the system is not intended to do X", the meaning of the change is immediately obvious to the casual observer.
With enough eyeballs, all code bugs are shallow.
As Eric Raymond observed, sheer numbers are an antidote for the complexity and opaqueness of code. The more people who are looking, the more likely you are to find someone who knows enough to see the cause.
With any eyeballs at all, all doc bugs are obvious.
Since documents are transparent, bugs are immediately discernable. Typos and broken links can be spotted by anyone. Invalid content can be identified by anyone who knows the product, or who attempts to follow the instructions.
Code changes have far_reaching consequences.
Depending on the degree of modularity in the system, a change in one place can have unforeseen consequences in other aspects of the system. It is therefore necessary to keep track of changes, so you know what caused things to break, and so you can undo the changes that caused the failure.
Documentation changes have no secondary effects.
Even if documentation is horrendously, totally wrong, it can have no impact elsewhere in the system (with the possible exception of a broken link, which is hardly a cause for alarm.) The product will not be broken as a result of the change, nor will user programs be in danger of failure. The doc bug may be unfortunate, but it has no real impact on other areas of the system.
Code is the product
The code will be run. If it has a problem, it will not run properly. Code bugs are therefore very expensive, and must be prevented.

Documentation is information about the product
Documentation provides information about how to use the product. In the infinitely remote worst cases, it could provide directions that make it impossible to use the product. But in that case, as well as all lesser cases, email lists, FAQs, forums, blogs, technical support, and word of mouth provide a remedy until the docs are fixed. Only the worst doc bugs rise to the level of minor code bugs.
All changes in a set must be backed out at one time.
If you made a change to 3 class files, undoing the change requires undoing all 3 changes. Undoing only two or one of them is virtually guaranteed to fail. It is therefore vital to identify all changes that were made as part of a set--especially due to the possibility of unforeseen consequences.
Changes can be backed out one by one.
If changes are made to three different documentation files, nothing absolutely requires them to be backed out together. One could be changed today, another next week, and the third a month after that. While perhaps not ideal, there are no significant penalties for making changes in that fashion.
Code bugs feed into Quality Assurance.
Every bug found in the system is an opportunity for adding an additional entry to the test suite--a suite which, by virtue of being automated, can be run as often as needed to prevent regressions.
Doc bugs are not handled by Quality Assurance.
There are no automated tests for docs, no automated way to prevent regressions. So doc bugs provide none of the salutary benefits that code bugs deliver, in that respect. (But docs are transparent, and easily corrected, so "regressions" are much less of an issue in any case.)
Code bugs almost always track important information.
Because code is opaque, the reason for unexpected behavior is typically unknown at the outset. The bug report makes a good place to discuss theories and devise isolation-tests. Once the problem is identified, there are typically multiple ways to solve the problem. The bug report provides a place to discuss those options and record the eventual selection.
Doc bugs rarely track important information
Documentation is transparent. When something is wrong, there is little if any doubt as to what the problem is. There are no tests to devise, and no options to discuss. There are rare exceptions, in which case a bug report can play a useful role, but, typically, a writer talks to a content expert, gets the necessary information, and writes up the changes.
Code bugs are a tiny part of the engineering process.
An engineer opens the bug and may even carry on a conversation within the context of the bug. But most the engineer's time is spent examining code to find the source of the problem, creating unit tests, fixing the code, and running the tests. Then the bug is closed. The time spent working with the bug report is a fraction of the time that is spent fixing the problem.
Doc bugs can be a huge part of the writing process.
The time spent processing a bug report and doing a workspace push can dwarf the time needed to make a change--especially when the bug report identifies a broken link or a typo. The work-to-reporting ratio for a writer is therefore the inverse of what it is for an engineer.
An engineer tends to work on one bug at a time.
Within any given workspace, it is almost always the case that an engineer will work on exactly one bug at a time. (If there are delays in the approval process, a second workspace may be created to work with another bug, but that workspace will also deal with only one bug at a time.)
A writer can, and should, work on many bugs at once.
If a document has a broken link, a typo, and a suggested revision, common sense suggests that all three changes should be made at one time. But the efficiency of that process is offset by the need to separately open and close three different bugs, which further magnifies the inverse work-to-reporting ratio.
Code bugs can help engineers manage their workflow.
A code bug can require a collection of synchronized changes in multiple files. The bug report can be used to identify the changes that have to be made, and act as checklist when the changes are implemented.
Doc bugs do not help writers manage their workflow.
For writers, the bugs-to-file ratio is inverted. One file may have many bugs. To work efficiently, the writer needs a list of all bugs against a given document, or in a given directory, sorted by document. But no extant bug tracking system has categories that are useful for that purpose.
Code bugs identify modules, using categories.
When a problem occurs, it is typically possible to identify the subsystem in which the error occurs, but the specific file or files that cause the problems, much less the lines in those files that need to be fixed. Investigation is required to make that assessment, and the result of that evaluation is recorded in the bug report before taking the next step.
Doc bugs identify files, categories of little help.
Because docs are transparent, bug reports identify a specific file, and typically a specific line in that file. But the category depth that is sufficient for code is of little or no help at all for docs. Because docs indentify specific files, the categories should be of unlimited depth. They could then duplicate the file's path, making it possible to sort for all bugs that pertain to a specific file, directory, or product. Absent that capability, bug categories provide little or no value for writers.

Your post is valid if you were writing for the Enquirer.
Documentation is a little more important for complex systems that users want to understand and use fully.
Then again, users don't expect much from documentation from open source.

Posted by C B on October 24, 2008 at 06:00 AM PDT #

C B wrote
> Docs are important
Couldn't agree more. If doc'n doesn't describe, then the coder never wrote it--because no one will ever know it's there. If the doc'n is wrong, it might as well not be there, because no one will ever use it.

I don't mean to imply that bugs are useless. There are plenty of reasons for filing them:

\* In the absence of other means, they give users a
way to provide feedback.

\* New engineering work causes a bug report to be filed,
to be sure doc'n gets done. (In which case, the
doc'n should have the same priority as the engineering
work. So if the work is critical to the release, then
the release does not ship until the information has
been fully transmitted to the writer and the final
docs reviewed.

\* If writers are too busy to react quickly, then the
bug provides a way to track the desired changed so
the information isn't lost.

\* Multiple documents have to change in a synchronized
way, and the writer wants one place to keep track
of the reasons for and implications of those changes.

So bugs are useful in a variety of circumstances. I'm against forcing them to be created and then closed for every single change a writer might want to make.

Since a single character change can cause an entire program to break, coders often get used to the mantra "a bug report for every change". For code, that mantra proves its value time and again, with the result that coders may get wedded to the idea.

But as I've tried to suggest, docs ain't code. If a writer corrects a typo, fixes a link, or changes some wording, the world does not come to a screeching halt if said writer gets it wrong. For that reason, the mantra is inappropriate for doc'n changes.

Posted by Eric Armstrong on October 26, 2008 at 07:34 AM PDT #

If doc bugs are obvious, because the doc is transparent, and if all doc bugs are about unimportant information, how is it that serious doc bugs appear in the final published documents?

You trivialize documentation and the documentation process. Reality is that documentation does require a quality review that includes fact checking, readability, clarity, consistency, and attention to the needs of the target audience. As with code, bug reports are a way to track problems, verify they are fixed correctly, analyze the process problems and patterns that lead to the creation and release of documentation problems, etc.

Producing quality documentation is more difficult than you give credit for, and the impact on sales, and increased support calls, etc. can be just as significant as code bugs.

Posted by Tex Texin on July 22, 2009 at 05:39 PM PDT #

Tex is right, you do trivialize documentation, and I would add that you miss just how dangerous this can be.

If the document in question is the operation manual for, say, a nuclear power plant, the consequences of "don't do X" rather than "do X" may be anything but trivial. In this instance the world might "come to a screeching halt if said writer gets it wrong", at least for the person following the instructions!

IMO this highlights that usage of bug reports and enhancement requests should be directly proportional to the potential consequences of a mistake.

Posted by Dantiumpro on January 29, 2010 at 07:34 PM PST #

Post a Comment:
  • HTML Syntax: NOT allowed

« July 2016