Vocabulary and documentation tool: tools.scaffold

The Factor project is on a documentation binge until all of the core and basis vocabularies are documented. I noticed, as anyone tasked to write a bunch of documenation would, that quite a bit of it could be automated. To begin writing docs for a vocabulary that already exists, you can just run the tool in it and edit the generated help file.

I actually ended up writing two tools -- one for the docs, and another to create new vocabularies with a simple command.

Scaffold tool to create new vocabularies

First, an aside. Factor's module system is based on a directory layout, with several vocabulary roots which are stored in the vocab-roots symbol.

( scratchpad ) USE: vocabs.loader vocab-roots get .
V{ "resource:core" "resource:basis" "resource:extra" "resource:work" }

Knowing this, you can now create a new vocabulary and boilerplate empty Factor files from inside of Factor and click the links to edit them in your favorite editor.

( scratchpad ) "extra" "alchemy" scaffold-vocab
Creating scaffolding for P" resource:extra/alchemy/alchemy.factor"
Creating scaffolding for P" resource:extra/alchemy/alchemy-tests.factor"
Creating scaffolding for P" resource:extra/alchemy/authors.txt"

( scratchpad ) "extra" "alchemy" scaffold-help
Creating scaffolding for P" resource:extra/alchemy/alchemy-docs.factor"

The scaffold tool is programmed not to overwrite files if they already exist.

Scaffold tool for documenation

Say I have a Factor word that turns lead into gold:

: lead>gold ( lead -- gold ) ... ;

The implementation of this word is left as an exercise to the reader. We don't have to understand how it works to document what it does.

Without an automation tool, you would begin by creating a new file (extra/alchemy/alchemy-docs.factor) in the same directory as your code file (extra/alchemy/alchemy.factor). What follows is the standard bolierplate for documentation. Notice that documenation goes in the same vocabulary as your actual code, but comes from different files on disk.

! Copyright (C) 2008 Doug Coleman.
! See http://factorcode.org/license.txt for BSD license.
USING: help.markup help.syntax ;
IN: alchemy

Every doc file has such a header based on the date, who you are, and the vocabulary name. With those three pieces of information, the above can be auto-generated. I added a new symbol developer-name in tools.scaffold that should be set to your name when writing docs.

Now let's look at the documenation for lead>gold:

HELP: lead>gold
{ $values { "lead" "boring lead" } { "gold" "shiny gold" } }
{ $description "Turns lead into gold." } ;

Notice that we know the word name, the first strings in each stack effect in the $values array, and the markup structure.

Here is the generated scafford:

HELP: lead>gold
{ $values { "lead" null } { "gold" null } }
{ $description } ;

This is much less wrist punishment than typing manually. The scaffold tool can even generate the documentation with their Factor types given correct code and standard type names like "quot" and "hashtable". Types that are not known are given the null class. Documenation correctness tools can later search for null objects and report them as errors, since no words should operate on this type.

I'll finish by including a snippet of the output of running the scaffold help generator on itself.

! Copyright (C) 2008 Doug Coleman.
! See http://factorcode.org/license.txt for BSD license.
USING: arrays help.markup help.syntax io.streams.string kernel strings words ;
IN: tools.scaffold

HELP: check-root
{ $values
     { "string" string }
     { "string" string } }
{ $description } ;

HELP: check-scaffold
{ $values
     { "vocab-root" "a vocabulary root string" } { "string" string }
     { "vocab-root" "a vocabulary root string" } { "string" string } }
{ $description } ;

HELP: check-vocab-name
{ $values
     { "string" string }
     { "string" string } }
{ $description } ;

HELP: developer-name
{ $description } ;

HELP: help.
{ $values
     { "word" word } }
{ $description } ;

HELP: lookup-type
{ $values
     { "string" string }
     { "object/string" null } { "?" "a boolean" } }
{ $description } ;

HELP: main-file-string
{ $values
     { "vocab" "a vocabulary specifier" }
     { "string" string } }
{ $description } ;

HELP: not-a-vocab-root
{ $values
     { "string" string } }
{ $description } ;

HELP: not-a-vocab-root?
{ $values
     { "object" object }
     { "?" "a boolean" } }
{ $description } ;

HELP: root?
{ $values
     { "string" string }
     { "?" "a boolean" } }
{ $description } ;

HELP: scaffold-authors
{ $values
     { "path" "a pathname string" } }
{ $description } ;

HELP: scaffold-copyright
{ $description } ;

HELP: tests-file-string
{ $values
     { "vocab" "a vocabulary specifier" }
     { "string" string } }
{ $description } ;

HELP: using
{ $description } ;

HELP: vocab>scaffold-path
{ $values
     { "vocab-root" "a vocabulary root string" } { "string" string }
     { "path" "a pathname string" } }
{ $description } ;

ARTICLE: "tools.scaffold" "tools.scaffold"
;

ABOUT: "tools.scaffold"

I hope this new vocabulary makes documenation a lot less tedious and error-prone. Now to go write some docs...