Slide 1

Slide 1 text

Eric Holscher Confoo Vancouver December 6, 2016 Understanding Documentation Systems

Slide 2

Slide 2 text

Who am I • Co-Founder of Read the Docs • Co-Founder of Write the Docs • I come from the Python world

Slide 3

Slide 3 text

Today • Understand why docs are important • Under the underlying model for documentation tools • Be able to reason more completely about how to extend doc tooling

Slide 4

Slide 4 text

@ericholscher Why Write Docs

Slide 5

Slide 5 text

@ericholscher You will be using your code in 6 months

Slide 6

Slide 6 text

@ericholscher You want people to use your code

Slide 7

Slide 7 text

@ericholscher It makes your code better

Slide 8

Slide 8 text

@ericholscher You want to be a better writer

Slide 9

Slide 9 text

@ericholscher Documentation Systems

Slide 10

Slide 10 text

Documentation Systems • Lightweight Markup Language • Templates • Command Line Interface • Common Output Formats (HTML, PDF)

Slide 11

Slide 11 text

Documentation Systems • Jekyll • Asciidoctor • Sphinx

Slide 12

Slide 12 text

Jekyll • Built on Markdown • A simpler execution model • Great for static sites, blogs, and documentation. • A great number of extensions, and support on GitHub Pages

Slide 13

Slide 13 text

Asciidoctor • Built on Asciidoc markup • Has powerful semantic constructs • Compatible with Docbook XML • Implementation in Ruby, but support for JVM, JS.

Slide 14

Slide 14 text

Sphinx • Built on reStructuredText Markup • Has powerful semantic constructs • Has a lot of primitives for documenting so ware • Has a lot of extensions, especially around testing & documentation source code

Slide 15

Slide 15 text

@ericholscher Lightweight Markup Languages

Slide 16

Slide 16 text

@ericholscher Lightweight Markup Language • Base format to generate other formats from • Readable as plain text • Works well with programmer tools • Works great in code comments/ docstrings

Slide 17

Slide 17 text

@ericholscher Semantic Meaning • The power of HTML & LWML • Semantics mean you are saying what something is, not how to display it • Once you know what it is, you can display it properly • “Separation of Concerns”

Slide 18

Slide 18 text

@ericholscher # HTML (Bad) issue 72 # HTML (Good) issue 72 # CSS (Good) .issue { text-format: bold; } Classic HTML Example

Slide 19

Slide 19 text

@ericholscher # Bad Warning: Don’t do this! # Good Don’t do this! # Best .. warning:: Don’t do this Classic RST Example

Slide 20

Slide 20 text

@ericholscher # Markdown Check out [PEP 8](https:// www.python.org/dev/peps/pep-0008/) # RST Check out :pep:`8` # Asciidoc See pep:8 to get started. Semantic Comparison

Slide 21

Slide 21 text

@ericholscher # Markdown This is an [idempotent](http:// docs.foo.com/glossary#term- idempotent] implementation. # RST This is an :term:`idempotent` implementation. # Asciidoc This is an term:idempotent implementation Semantic Comparison

Slide 22

Slide 22 text

@ericholscher See our image :ref[scatter plot] to see more information. :: include{file=other-file.md} CommonMark Proposal https://talk.commonmark.org/t/generic-directives-plugins- syntax/444

Slide 23

Slide 23 text

@ericholscher Semantic Markup • Shows the intent of your words • Works across output formats • You can style warnings differently in HTML, PDF, ePub, etc.

Slide 24

Slide 24 text

@ericholscher Extendability

Slide 25

Slide 25 text

@ericholscher Inline Markup • Anything that is included in the page content itself • Used for embedding things into the rendered output

Slide 26

Slide 26 text

@ericholscher Inline Markup • Looks like :role:`target` • :pep:`8` will create a link to PEP 8

Slide 27

Slide 27 text

@ericholscher Coding Reference ---------------- Generally we follow :pep:`8` in our code. However, we also have our own :doc:`style-guide` with exceptions. Inline Markup Example

Slide 28

Slide 28 text

@ericholscher Inline Markup Example

Slide 29

Slide 29 text

@ericholscher Page Level Markup • Allows you to nest content inside of them • Great for reference endpoints

Slide 30

Slide 30 text

@ericholscher Page Level Markup • .. directive-name:: • Main source of extendability • A lot of Sphinx’s power comes through Directives

Slide 31

Slide 31 text

@ericholscher .. code-block:: python :linenos: from datetime import datetime time = datetime.now() print time Page Level Example

Slide 32

Slide 32 text

@ericholscher Page Level Example

Slide 33

Slide 33 text

Templates • Allow for extending with specific logic in the markup language • Jinja & Liquid are the major ones

Slide 34

Slide 34 text

{% for speaker in talk.speakers %} * {{ speaker.name }} {% endfor %} Templates

Slide 35

Slide 35 text

reStructuredText Implementation

Slide 36

Slide 36 text

Parts • Reader • Parser • Transformer • Writer

Slide 37

Slide 37 text

How it fits together

Slide 38

Slide 38 text

Reader • Get input and read it into memory • Quite simple, generally don’t need to do much

Slide 39

Slide 39 text

Title ===== Paragraph. Words that have **bold in them**. Reader Example

Slide 40

Slide 40 text

[u'Title', u'=====', u'', u'Paragraph.', u'', u'Words that have **bold in them**.'] Reader Example

Slide 41

Slide 41 text

Reader’s are useful for adding non-filesystem types of input (StringIO, Network)

Slide 42

Slide 42 text

Parser • Takes the input and actually turns it into a Doctree • Handles directives, inline markup, etc. • RST is the only parser implemented in Docutils • Implemented with a lined-based recursive state machine

Slide 43

Slide 43 text

Doctree • AST for Docutils • Source of Truth • Hierarchy with a document at the root node

Slide 44

Slide 44 text

Title ===== Paragraph. Words that have **bold in them**. Doctree Example

Slide 45

Slide 45 text

Title Paragraph. Words that have bold in them Doctree Example

Slide 46

Slide 46 text

Parsers are used for implementing new RST features or adding new markup languages

Slide 47

Slide 47 text

Transformer • Take the doctree and modify it in place • Allows for full knowledge of the content • Table of Contents • Implemented by traversing nodes

Slide 48

Slide 48 text

Title Paragraph. Words that have bold in them Transform Example

Slide 49

Slide 49 text

Title Words that have bold in them Paragraph. Transform Example

Slide 50

Slide 50 text

Writer • Takes the Doctree and writes it to actual files • HTML, XML, PDF, etc. • Implemented with a Visitor pattern

Slide 51

Slide 51 text

How it fits together

Slide 52

Slide 52 text

Extensions

Slide 53

Slide 53 text

Parser • rST • Markdown

Slide 54

Slide 54 text

Markdown Parser • Uses recommonmark • Translates Commonmark Node’s into Docutils Node’s

Slide 55

Slide 55 text

def paragraph(self, block): p = nodes.paragraph() p.line = block.start_line append_inlines(p, block.inline_content) self.current_node.append(p) Markdown Parser

Slide 56

Slide 56 text

## Markdown Header Hey There Markdown Parser

Slide 57

Slide 57 text

Markdown Header Hey There Markdown Parser

Slide 58

Slide 58 text

Transformer • Table of contents

Slide 59

Slide 59 text

TOC Transformer • Enabled with `.. contents::` directive • Adds a pending node during parsing • Gets properly set with a transform

Slide 60

Slide 60 text

.. contents:: TOC TOC Transformer

Slide 61

Slide 61 text

TOC .. internal attributes: .transform: docutils.transforms.parts.Contents .details: TOC Transformer

Slide 62

Slide 62 text

TOC Getting Started TOC Transformer

Slide 63

Slide 63 text

Sphinx

Slide 64

Slide 64 text

Sphinx • Builds on top of the standard docutils concepts • Add it’s own abstractions, but uses the same docutils machinery underneath • Goes from single page to project

Slide 65

Slide 65 text

Sphinx Components • Application • Environment • Builder

Slide 66

Slide 66 text

Sphinx Application • Main level of orchestration for Sphinx • Handles configuration & building • Sphinx()

Slide 67

Slide 67 text

Sphinx Environment • Keeps state for all the files for a project • Serialized to disk in between runs • Works as a cache

Slide 68

Slide 68 text

Sphinx Builder • Handles generating output • Where templates & styles are implemented • Allow customization of specific elements

Slide 69

Slide 69 text

Sphinx layers on top of RST to provide full project support

Slide 70

Slide 70 text

Take Aways

Slide 71

Slide 71 text

Knowing how your docs tools work is part of your job

Slide 72

Slide 72 text

Think about your intended use case and outcomes before picking a tool

Slide 73

Slide 73 text

O en times you can save a lot of effort by writing simple extensions to your tools

Slide 74

Slide 74 text

Learn how other languages solve problems, then steal liberally :)

Slide 75

Slide 75 text

Questions? • @ericholscher • [email protected] • Come talk to me around the conference