Trials of Documenting JavaScript Source Code

This should be a solved problem, by now. It's 2014, for crying out loud! Documenting source code in other languages is pretty much trivial; for Python there's Sphinx, Ruby has RDoc, for Java you have JavaDoc, and then there's Doxygen if you need to document C/C++ code. These are all pretty much de facto standards in their respective realms. What do we have for JavaScript?

A horrible wart called JSDoc3.

I hate JSDoc3. It's slow, the default template is unacceptably bad, it's difficult to create links between symbols in a semantic way, formatting requires writing HTML junk into your doc comments, ...

JSDoc3 is so bad, there are tons of replacements available:

• Docco
• dox-foundation
• Doxx
• YUIDoc
• dox-docco

And plenty of forks that were omitted. (I've only included generators written in JavaScript for this list. I've also seen Sphinx, JSDuck, and some others that I won't go into, because they cannot be used as node-modules.)

Docco doesn't support structured comments, and it's really designed for micro projects or tutorials (see Jasmine documentation for an example), so that's out.

dox-foundation is a good start with markdown support, but organization is a major issue (everything is sorted by file name!) and it still doesn't solve the problem with linking symbols.

Doxx is also nice (and is a fork of dox-foundation), but it's still not good at organizing generated documentation, and linking between symbols is still a pain. And what's with the jade templates? I like to write my HTML in HTML.

YUIDoc looks just as big and clunky as JSDoc3, and its default template looks like something out of Microsoft or IBM.

dox-docco is similar to Docco, but with structured comment support. It's also best for tutorials.


Dissatisfied with everything available, I decided to use dox and Prism (the same components behind dox-foundation, Doxx, and dox-docco) and combine them with doT to create an incredibly fast and flexible documentation generator that will do everything I need for one of my personal projects. Styling is based on Foundation, a la dox-foundation.

Here's what it looks like so far:

node-capstone documentation

This is a screenshot showing the WIP documentation for a project I've been hacking on for the past few weeks. (I will blog about that project later.) The project is a node module, and I wanted an easy way to generate the documentation that meets the following criteria:

• It has to be blazing fast! No more waiting 30-seconds to spin up a JVM, please.
• It must also be lightweight, but not limiting; easy to extend.
• It should support JSDoc3-style tags in block comments.
• It must support styling within documentation, preferably without HTML soup.
• It must be capable of organizing documentation in a more sane manner than sorted-by-filename.
• It must provide links to contextually relevant information in a semantic way.
• The theme must not make my eyes bleed.

I think I've accomplished all of these goals, so far. It's a tiny JavaScript app that just does some comment parsing, symbol organization, and template rendering (with a really fast renderer). The template renderer, doT, is awesome because the template itself can run JavaScript, meaning it can do some of the heavy lifting (which is important as you will see later).

dox parses JSDoc3-style block comment tags, and integrates markdown, so that's perfect.

After comments are parsed, symbols are organized; first by kind (classes, functions, constants, miscellaneous). On the TODO List: child symbols below the four top-level categories will be further organized by kind (again), then sorted alphabetically.

Data types are already linked properly (the arguments table for INSN_OFFSET() in the screenshot has a type linking to a class) and automatically; there's no special documentation syntax for it! It just works.

The theme is easy on the eyes; it's pretty, it doesn't get in the way with obnoxious frames, drop-down menus, or accordions. There are no bouncy animations, giant titles, or logos; just the juicy meat of the documentation! Syntax highlighting is all there. The full source code for each symbol is included, hidden by default, and automatically highlighted as well; in the screenshot, source code for INSN_OFFSET() is shown.

All this was built in just the last three days or so, and I've been creating it as a grunt task from the start, so hopefully it can be reused again on other projects. The grunt task uses metadata from the project's package.json, and can get additional information (or overrides) from the task options.

Design of the Generator

After parsing comments with dox, it does some basic symbol organization to build a tree structure that will represent the final documentation output. The tree is simply passed on to the template renderer. The template is responsible for inferring types and generating proper links between symbols; the renderer and parsers do not care about context. Doing the type inference late in the rendering allows the template to choose link destinations (not just DOM structure and styling) to keep the documentation comment semantics separate from the documentation template structure. Only the template knows where the template is putting symbols!

At present it is not ready to take on a project the size of melonJS. In fact, it's not even complete enough to fully document my little node module! You'll even notice in the screenshot there are three kind-of-broken links at the end of the side navigation bar. ;) Only the functions and main pages have templates.

It has been a very pleasant side project, though. Of all the components I'm using, the one with the steepest learning curve has been Foundation ... and that's just a bunch of CSS! dox makes comment parsing dead simple, and provides enough context to create beautiful documentation. Prism was absolutely zero-effort to integrate (after installing a dox fork that uses marked for markdown parsing). And doT is extremely versatile; it's almost ... magical. ;) The last piece, which I haven't mentioned yet, is Font Awesome. Four of the five icons I've used can be seen in the screenshot. The last icon I'm using is for Twitter, and it appears in the page footer if you've specified a twitter handle in the grunt task options. How's that for awesome?

Next Steps

I actually have a small TODO list for this side project, and it happens to be a larger list than the node module which required documentation in the first place! One thing that would be handy is printing the file name and line number alongside the source code, but this will require extra patches to dox; may as well fork it at this point.

Another thing to do is make the side navigation bar "sticky", so I don't have to scroll to the top of the page each time I want to visit a different symbol. This is actually more important than it sounds. ;)

I have not started on the "class", "constants", or "miscellaneous" pages at all! But considering the "functions" page was blank just yesterday (and now it looks like the screenshot) I would say it isn't a lot of trouble to do each page. To be honest, I was expecting it to take longer for some reason.

And lastly, to actually package it as a grunt plugin. Right now it's just embedded into the node module's repo, where it doesn't really belong! That should just be a matter of running grunt-init. But then there's the task of documenting it! Hahaha!