Eric Holscher
Confoo Vancouver
December 6, 2016
Understanding
Documentation Systems
Slide 2
Slide 2 text
Who am I
• Co-Founder of Read the Docs
• Co-Founder of Write the Docs
• I come from the Python world
Slide 3
Slide 3 text
Today
• Understand why docs are important
• Under the underlying model for
documentation tools
• Be able to reason more completely
about how to extend doc tooling
Slide 4
Slide 4 text
@ericholscher
Why Write Docs
Slide 5
Slide 5 text
@ericholscher
You will be using your
code in 6 months
Slide 6
Slide 6 text
@ericholscher
You want people to use
your code
Slide 7
Slide 7 text
@ericholscher
It makes your code
better
Slide 8
Slide 8 text
@ericholscher
You want to be a better
writer
Slide 9
Slide 9 text
@ericholscher
Documentation Systems
Slide 10
Slide 10 text
Documentation Systems
• Lightweight Markup Language
• Templates
• Command Line Interface
• Common Output Formats (HTML, PDF)
Slide 11
Slide 11 text
Documentation Systems
• Jekyll
• Asciidoctor
• Sphinx
Slide 12
Slide 12 text
Jekyll
• Built on Markdown
• A simpler execution model
• Great for static sites, blogs, and
documentation.
• A great number of extensions, and
support on GitHub Pages
Slide 13
Slide 13 text
Asciidoctor
• Built on Asciidoc markup
• Has powerful semantic constructs
• Compatible with Docbook XML
• Implementation in Ruby, but support
for JVM, JS.
Slide 14
Slide 14 text
Sphinx
• Built on reStructuredText Markup
• Has powerful semantic constructs
• Has a lot of primitives for documenting
so ware
• Has a lot of extensions, especially
around testing & documentation source
code
Slide 15
Slide 15 text
@ericholscher
Lightweight
Markup Languages
Slide 16
Slide 16 text
@ericholscher
Lightweight Markup
Language
• Base format to generate other formats
from
• Readable as plain text
• Works well with programmer tools
• Works great in code comments/
docstrings
Slide 17
Slide 17 text
@ericholscher
Semantic Meaning
• The power of HTML & LWML
• Semantics mean you are saying what
something is, not how to display it
• Once you know what it is, you can
display it properly
• “Separation of Concerns”
Slide 18
Slide 18 text
@ericholscher
# HTML (Bad)
issue 72
# HTML (Good)
issue 72
# CSS (Good)
.issue { text-format: bold; }
Classic HTML Example
Slide 19
Slide 19 text
@ericholscher
# Bad
Warning: Don’t do this!
# Good
Don’t do this!
# Best
.. warning:: Don’t do this
Classic RST Example
Slide 20
Slide 20 text
@ericholscher
# Markdown
Check out [PEP 8](https://
www.python.org/dev/peps/pep-0008/)
# RST
Check out :pep:`8`
# Asciidoc
See pep:8 to get started.
Semantic Comparison
Slide 21
Slide 21 text
@ericholscher
# Markdown
This is an [idempotent](http://
docs.foo.com/glossary#term-
idempotent] implementation.
# RST
This is an :term:`idempotent`
implementation.
# Asciidoc
This is an term:idempotent
implementation
Semantic Comparison
Slide 22
Slide 22 text
@ericholscher
See our image :ref[scatter plot] to
see more information.
:: include{file=other-file.md}
CommonMark Proposal
https://talk.commonmark.org/t/generic-directives-plugins-
syntax/444
Slide 23
Slide 23 text
@ericholscher
Semantic Markup
• Shows the intent of your words
• Works across output formats
• You can style warnings differently in
HTML, PDF, ePub, etc.
Slide 24
Slide 24 text
@ericholscher
Extendability
Slide 25
Slide 25 text
@ericholscher
Inline Markup
• Anything that is included in the page
content itself
• Used for embedding things into the
rendered output
Slide 26
Slide 26 text
@ericholscher
Inline Markup
• Looks like :role:`target`
• :pep:`8` will create a link to PEP 8
Slide 27
Slide 27 text
@ericholscher
Coding Reference
----------------
Generally we follow :pep:`8` in our code.
However, we also have our own :doc:`style-guide` with
exceptions.
Inline Markup Example
Slide 28
Slide 28 text
@ericholscher
Inline Markup Example
Slide 29
Slide 29 text
@ericholscher
Page Level Markup
• Allows you to nest content inside of
them
• Great for reference endpoints
Slide 30
Slide 30 text
@ericholscher
Page Level Markup
• .. directive-name::
• Main source of extendability
• A lot of Sphinx’s power comes through
Directives
Slide 31
Slide 31 text
@ericholscher
.. code-block:: python
:linenos:
from datetime import datetime
time = datetime.now()
print time
Page Level Example
Slide 32
Slide 32 text
@ericholscher
Page Level Example
Slide 33
Slide 33 text
Templates
• Allow for extending with specific logic
in the markup language
• Jinja & Liquid are the major ones
Slide 34
Slide 34 text
{% for speaker in talk.speakers %}
* {{ speaker.name }}
{% endfor %}
Templates
Slide 35
Slide 35 text
reStructuredText
Implementation
Slide 36
Slide 36 text
Parts
• Reader
• Parser
• Transformer
• Writer
Slide 37
Slide 37 text
How it fits together
Slide 38
Slide 38 text
Reader
• Get input and read it into memory
• Quite simple, generally don’t need to do
much
Slide 39
Slide 39 text
Title
=====
Paragraph.
Words that have **bold in them**.
Reader Example
Slide 40
Slide 40 text
[u'Title', u'=====', u'', u'Paragraph.', u'', u'Words
that have **bold in them**.']
Reader Example
Slide 41
Slide 41 text
Reader’s are useful for
adding non-filesystem types
of input (StringIO, Network)
Slide 42
Slide 42 text
Parser
• Takes the input and actually turns it
into a Doctree
• Handles directives, inline markup, etc.
• RST is the only parser implemented in
Docutils
• Implemented with a lined-based
recursive state machine
Slide 43
Slide 43 text
Doctree
• AST for Docutils
• Source of Truth
• Hierarchy with a document at the root
node
Slide 44
Slide 44 text
Title
=====
Paragraph.
Words that have **bold in them**.
Doctree Example
Slide 45
Slide 45 text
Title
Paragraph.
Words that have
bold in them
Doctree Example
Slide 46
Slide 46 text
Parsers are used for
implementing new RST
features or adding new
markup languages
Slide 47
Slide 47 text
Transformer
• Take the doctree and modify it in place
• Allows for full knowledge of the
content
• Table of Contents
• Implemented by traversing nodes
Slide 48
Slide 48 text
Title
Paragraph.
Words that have
bold in them
Transform Example
Slide 49
Slide 49 text
Title
Words that have
bold in them
Paragraph.
Transform Example
Slide 50
Slide 50 text
Writer
• Takes the Doctree and writes it to
actual files
• HTML, XML, PDF, etc.
• Implemented with a Visitor pattern
Sphinx
• Builds on top of the standard docutils
concepts
• Add it’s own abstractions, but uses the
same docutils machinery underneath
• Goes from single page to project