Hyperlinked, externalized source code documentation [closed]

Question

Why do we still embed natural language descriptions of source code (i.e., the reason why a line of code was written) within the source code, rather than as a separate document?

Given the expansive real-estate afforded to modern development environments (high-resolution monitors, dual-monitors, etc.), an IDE could provide semi-lock-step panels wherein source code is visually separated from -- but intrinsically linked to -- its corresponding comments. For example, developers could write source code comments in a hyper-linked markup language (linking to additional software requirements), which would simultaneously prevent documentation from cluttering the source code.

What shortcomings would inhibit such a software development mechanism?

A mock-up to help clarify the question:

When the cursor is at a particular line in the source code (shown with a blue background, above), the documentation that corresponds to the line at the cursor is highlighted (i.e., distinguished from the other details). As noted in the question, the documentation would stay in lock-step with the source code as the cursor jumps through the source code. A hot-key could switch between "documentation mode" and "development mode".

Potential advantages include:

• More source code and more documentation on the screen(s) at once
• Ability to edit documentation independently of source code (regardless of language?)
• Write documentation and source code in parallel without merge conflicts
• Real-time hyperlinked documentation with superior text formatting
• Quasi-real-time machine translation into different natural languages
• Every line of code can be clearly linked to a task, business requirement, etc.
• Documentation could automatically timestamp when each line of code was written (metrics)
• Dynamic inclusion of architecture diagrams, images to explain relations, etc.
• Single-source documentation (e.g., tag code snippets for user manual inclusion).

Note:

• The documentation window can be collapsed
• Workflow for viewing or comparing source files would not be affected
• How the implementation happens is a detail; the documentation could be:
  • kept at the end of the source file;
  • split into two files by convention (filename.c, filename.c.doc); or
  • fully database-driven
• By hyperlinked documentation, I mean linking to external sources (such as StackOverflow or Wikipedia) and internal documents (i.e., a wiki on a subdomain that could cross-reference business requirements documentation) and other source files (similar to JavaDocs).

Related thread: What's with the aversion to documentation in the industry?

Answer 1

This problem bothers me all the time, and I just got an idea on the direction we should be going to try and solve it (thats how I found this question).

I think the linked documentation problem was thought wrong when we decided to include the user documentation in the source code. Like docco does.

First of all, lets differentiate code comments from user documentation .

Code comments normally are in its best when they are short, super short in fact, so I can actually read the code that does the stuff without having to see some poetry of why and how it works.

User comments are intended for people trying to use your library/API, and may include examples of use, explanation of why it was implemented like that, or instructions on how to extend the library. This kind of comments tend to be really verbose.

I do agree on the fact that documentation should be written in plain text so is not vendor fixed and its easy to add in a VCS. But I think that keeping the user documentation in the source file was a big mistake for at least two reasons:

• Harder to read code
• Not flexible enough documentation (suppose I need two documentation pages using the same example code or having one documentation page needing to interleave code from two different source files).

So why do we want to have it in the same file? Well, nobody wants to have their documentations out of sync from the code. But we do it anyways, and specially now a days with the big success of Markdown. Which I think is on the right path, but if falls short, waaay short.

When we interleave code comment with user comment, we have a 2 way binding. That allows us to easily see which comment corresponds to what part of the code. We can also see if some code is undocumented. What it doesn't offer, is a way to see if the comment is updated or not, and that happens a lot when your code is hard to read (because the documentation made it ugly).

What if instead of having a 2 way binding, we have a one way?. Documentation pointing to code. We could have markdown code with special commands like:

Some documentation right here that explains the following code:
   @include_file <path/to/some/file>:<line_start>-<line_end>
or
   @include_file <path/to/some/file>
     @starts_with "some regexp or literal text to search"
     @ends_with "another regexp"
or
   @include_file <path/to/some/file>
     @class MyClass
     @method foo
or any combination or way of linking you could imagine

We can even have semantic in the directives:
   @explain_code <path/and/same/of/above>
   @example_code <path/and/same/of/above>
   @performance_notice <path/and/same/of/above>

Which would do basically the same, but it adds the intention of why
do we want to add this code in the first place, which could be 
used different by an IDE. 

This has the benefit of being markup with some additions, and with the proper tools, we could add more value to it.

Imagine this one way binding with tools like grunt (even with the watch task). You could detect when a source file change, see what documentation files depended on it and warn the user (or mark it out somewhere) if the code that was commented had change.

Answer 2

404 - Page Not Found

When working with code you don't want you comments to get lost and that's what will happen when you seperate code and comments into seperate documents.

Also, keeping up versioning between your comment document and code document will cause more pain then gain.

Some of the suggestions you make I really like but could easily be implemented while also having code and comments in 1 file.

Answer 3

Possible disadvantages that I see:

• You need a special editor that implements this feature

• The code is not just plain text anymore, easy to manipulate and commit to VCS-es

• You need twice more screen width to work with the code

As for your arguments:

More source code and more documentation on the screen(s) at once

But can be inconvenient if you want to view two files side-by-side.

Ability to edit documentation independently of source code (regardless of language?)

I would argue that it is actually a disadvantage. I personally try to keep the documentation as close as possible to the code, so that it does not become outdated.

Write documentation and source code in parallel without merge conflicts

Again, possibly a disadvantage. If your docs are deeply interleaved with code, how can you edit them independently?

Real-time hyperlinked documentation with superior text formatting

If it is in the code, it is already real-time ;) As for the hyperlinks, jumping to definition is already implemented in most IDEs.

Quasi-real-time machine translation into different natural languages

I don't see why you can't do that with regular comments/docstrings.

Every line of code can be clearly linked to a task, business requirement, etc.

This I'm uncertain about... Can't it be achieved with regular comments?

Documentation could automatically timestamp when each line of code was written (metrics)

Don't VCS-es provide this kind of information already?

Having said this, I quite like the layout itself — but I do not see the need of changing the file format, it is not that difficult to generate it from regular comments. There's a bunch of documentation generators which do that, e.g. Docco and its successors, like Pycco or Marginalia.

Answer 4

Firstly, the doc comments need to go with the code - moving them elsewhere only makes things incredibly difficult to handle for practically zero gain. So why bother!

What could be done though is to take those embedded comments and hide them in the editor, showing them in a bubble or tooltip or whatnot as needed. I would hope that such an approach might encourage people to write much more documentation to the code - eg, a description of a class could go with the class rather than in an external design document.

You can currently embed hyperlinks in code comments, and you can generate documents from the code using tools like Doxygen or Sphinx. I guess it just would take some fancy extension tot he code editor to better support these tools.

I wouldn't link any lines of code to a bug tracker or requirements tool, that's a better job for your SCM. Then you can see code edits per commit that are linked to a task. I wouldn't integrate the documentation stored in the code against the bug tracker either - you'd be screwed if you ever wanted to migrate to a new one (hmm, I can see this feature being added to TFS right now) or if you lost your commit history (which happens)

Answer 5

Besides what @Bogdan already states, I would add a few:

• I configured my IDE to always have 2 files at once. With the feature you are suggesting, I would basically need 2 monitors to see the same amount of information, and 3 to do what I'm doing now with 2.
• While browsing through a file, you don't immediately see the comments, and if you don't know exactly what you are looking for, it's very difficult to find it.
• While searching through a file, I can't search through comments directly (or as easily).
• I sometimes need to do various tests / write short pieces of code on the live server, through ssh. Although the functionality you are saying could be integrated with VIM or other command line editors - most likely there would be pretty big problems

• Most IDEs support collapsing code/comments, with the end results being the following:

  Instead of the normal:

  

Make the code more readable, in case you don't need the comments.

Answer 6

Source code and documentation the way it should be done.

http://jasmine.github.io/2.3/introduction.html