Better Documentation Through Automation: Creating docutils & Sphinx Extensions

Transcript

Better Documentation Through Automation:
Creating docutils & Sphinx Extensions
Doug Hellmann
@doughellmann
PyCon 2013
Saturday, March 16, 13

View Slide

Markup Language

View Slide

Markup Language
Extensible

View Slide

Sphinx
Application

View Slide

Build
Environment
Sphinx
Application
.rst

View Slide

Build
Environment
docutils
parser
Sphinx
Application
.rst

View Slide

Build
Environment
docutils
parser
Sphinx
Builder
Sphinx
Application
.rst .pdf .html

View Slide

section
title paragraph
#text emphasis #text literal
#text
#text
#text

View Slide

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam
vulputate elementum lectus a viverra. Vivamus hendrerit egestas
lacinia. Proin tellus lectus, scelerisque vitae rutrum eget,
ultrices vel metus. Quisque :ref:`target-name` pharetra lorem
vehicula lorem posuere molestie cursus ipsum ornare. Class
aptent taciti sociosqu ad litora torquent per conubia nostra,
per inceptos himenaeos. Maecenas eu sapien at tellus suscipit
aliquet :pep:`8` ut non tellus. Nulla facilisis bibendum dolor,
quis mattis urna posuere eget. Nulla quis dui id augue faucibus
varius. Nam eu leo quam. Fusce :term:`condimentum` placerat nisi
nec vestibulum. Duis tortor sapien, commodo ac faucibus
nec :rfc:`1822`, rutrum quis urna. Pellentesque cursus facilisis
odio, id aliquam sem commodo vestibulum. Proin eget velit sed
justo ultricies vulputate.

View Slide

.. seealso::
PyEnchant_
Python interface to enchant_.
:ref:`project-sphinxcontrib-spelling`
Project home page for the spelling checker.
sphinxcontrib_
BitBucket repository for sphinxcontrib-spelling and
several other Sphinx extensions.
.. include:: example.py
:literal:
:start-after: # end-of-header-comment
.. image:: figure.png

View Slide

2.6
- Fixed a problem with hook script line endings under Cygwin
(Ìssue 68 virtualenvwrapper/issue/68>`_).
- Updated documentation to include a list of the compatible
shells
(:ref:`supported-shells`) and Python versions
(:ref:`supported-versions`) (Ìssue 70 bitbucket.org/dhellmann/virtualenvwrapper/issue/70>`_).
- Fixed installation dependency on virtualenv (Ìssue 60
`_).
- Fixed the method for determining the Python version so it
works
under Python 2.4 (Ìssue 61 dhellmann/virtualenvwrapper/issue/60>`_).
- Converted the test infrastructure to use `tox
`_ instead of home-
grown
scripts in the Makefile.

View Slide

2.6
shells
`_).
works
`_ instead of home-
grown

View Slide

2.6
shells
`_).
works
`_ instead of home-
grown

`Issue 68 `_

`Issue 68 `_

`Issue 68 `_

`68`
`Issue 68 `_
:bbissue:

View Slide

from docutils import nodes, utils
from docutils.parsers.rst.roles import set_classes
def bbissue_role(name, rawtext, text, lineno, inliner, options={},
content=[]):
"Link to a BitBucket issue."
try:
issue_num = int(text)
if issue_num <= 0:
raise ValueError
except ValueError:
msg = inliner.reporter.error(
'"%s" is an invalid bug id.' % text,
line=lineno)
prb = inliner.problematic(rawtext, rawtext, msg)
return [prb], [msg]
app = inliner.document.settings.env.app
node = make_link_node(rawtext, app, 'issue',
text, options)
return [node], []
def make_link_node(rawtext, app, type, slug, options):
"Create a link to a BitBucket resource."
try:
base = app.config.bitbucket_project_url
if not base:

View Slide

content=[]):
try:
if issue_num <= 0:
raise ValueError
except ValueError:
line=lineno)
return [prb], [msg]
text, options)
return [node], []
try:
if not base:

View Slide

content=[]):
try:
if issue_num <= 0:
raise ValueError
except ValueError:
line=lineno)
return [prb], [msg]
text, options)
return [node], []
try:
if not base:

View Slide

content=[]):
try:
if issue_num <= 0:
raise ValueError
except ValueError:
line=lineno)
return [prb], [msg]
text, options)
return [node], []
try:
if not base:

View Slide

content=[]):
try:
if issue_num <= 0:
raise ValueError
except ValueError:
line=lineno)
return [prb], [msg]
text, options)
return [node], []
try:
if not base:

View Slide

content=[]):
try:
if issue_num <= 0:
raise ValueError
except ValueError:
line=lineno)
return [prb], [msg]
text, options)
return [node], []
try:
if not base:

View Slide

text, options)
return [node], []
try:
if not base:
raise AttributeError
except AttributeError, err:
raise ValueError('bitbucket_project_url not set (%s)' %
str(err))
slash = '/' if base[-1] != '/' else ''
ref = base + slash + type + '/' + slug + '/'
set_classes(options)
node = nodes.reference(rawtext,
type + ' ' + utils.unescape(slug),
refuri=ref,
**options)
return node
def setup(app):
"Install the plugin."
app.info('Initializing BitBucket plugin')
app.add_role('bbissue', bbissue_role)

View Slide

text, options)
return [node], []
try:
if not base:
str(err))
refuri=ref,
**options)
return node
def setup(app):

View Slide

text, options)
return [node], []
try:
if not base:
str(err))
refuri=ref,
**options)
return node
def setup(app):

View Slide

text, options)
return [node], []
try:
if not base:
str(err))
refuri=ref,
**options)
return node
def setup(app):

View Slide

text, options)
return [node], []
try:
if not base:
str(err))
refuri=ref,
**options)
return node
def setup(app):

View Slide

refuri=ref,
**options)
return node
def setup(app):
app.add_config_value('bitbucket_project_url', None, 'env')
return

View Slide

refuri=ref,
**options)
return node
def setup(app):
return

View Slide

refuri=ref,
**options)
return node
def setup(app):
return

View Slide

2.6
(:bbissue:`68`).
- Updated documentation to include a list of the
compatible shells
(:ref:`supported-versions`) (:bbissue:`70`).
- Fixed installation dependency on virtualenv (:bbissue:`60`).
- Fixed the method for determining the Python version so
it works under Python 2.4 (:bbissue:`61`).
`_ instead of
home-grown

View Slide

2.6
(:bbissue:`68`).
- Updated documentation to include a list of the
compatible shells
(:ref:`supported-versions`) (:bbissue:`70`).
- Fixed installation dependency on virtualenv (:bbissue:`60`).
- Fixed the method for determining the Python version so
it works under Python 2.4 (:bbissue:`61`).
`_ instead of
home-grown

View Slide

2.6
• Fixed a problem with hook script line endings under
Cygwin (issue 68).
• Updated documentation to include a list of the
compatible shells (Supported Shells) and Python
versions (Python Versions) (issue 70).
• Fixed installation dependency on virtualenv (issue 60).
• Fixed the method for determining the Python version so
it works under Python 2.4 (issue 61).
• Converted the test infrastructure to use tox instead of
home-grown scripts in the Makefile.

View Slide

.. name:: arguments
:option: value
:option: another-value
body line
body line

View Slide

.. name:: arguments
:option: value
body line
body line

View Slide

.. name:: arguments
:option: value
body line
body line

View Slide

.. name:: arguments
:option: value
body line
body line

View Slide

.. sqltable:: List of Users
:connection_string: sqlite:///sampledata.db
select
name as 'Name',
email as 'E-mail'
from
users
order by
Name asc

View Slide

select
name as 'Name',
email as 'E-mail'
from
users
order by
Name asc

View Slide

select
name as 'Name',
email as 'E-mail'
from
users
order by
Name asc

View Slide

select
name as 'Name',
email as 'E-mail'
from
users
order by
Name asc

View Slide

from docutils.parsers.rst.directives.tables import Table
class SQLTable(Table):
option_spec = {'widths': directives.positive_int_list,
'class': directives.class_option,
'name': directives.unchanged,
'connection_string':directives.unchanged,
}
def run(self):
env = self.state.document.settings.env
app = env.app
config = app.config
# Make sure we have some content, which for now we
# assume is a query.
if not self.content:
error = self.state_machine.reporter.error(
'No query in sqltable directive',
nodes.literal_block(self.block_text,
self.block_text),
line=self.lineno)
return [error]
# Connect to the database

View Slide

}
def run(self):
app = env.app
config = app.config
self.block_text),
line=self.lineno)
return [error]

View Slide

}
def run(self):
app = env.app
config = app.config
self.block_text),
line=self.lineno)
return [error]

View Slide

}
def run(self):
app = env.app
config = app.config
self.block_text),
line=self.lineno)
return [error]

View Slide

}
def run(self):
app = env.app
config = app.config
self.block_text),
line=self.lineno)
return [error]

View Slide

}
def run(self):
app = env.app
config = app.config
self.block_text),
line=self.lineno)
return [error]

View Slide

}
def run(self):
app = env.app
config = app.config
self.block_text),
line=self.lineno)
return [error]

View Slide

}
def run(self):
app = env.app
config = app.config
self.block_text),
line=self.lineno)
return [error]

View Slide

def run(self):
app = env.app
config = app.config
self.block_text),
line=self.lineno)
return [error]
connection_string = self.options.get(
'connection_string',
config.sqltable_connection_string,
)
app.info('Connecting to %s' % connection_string)
engine = sqlalchemy.create_engine(connection_string)
# Run the query
query = '\n'.join(self.content)
app.info('Running query %r' % query)

View Slide

def run(self):
app = env.app
config = app.config
self.block_text),
line=self.lineno)
return [error]
)
# Run the query

View Slide

def run(self):
app = env.app
config = app.config
self.block_text),
line=self.lineno)
return [error]
)
# Run the query

View Slide

line=self.lineno)
return [error]
)
# Run the query
results = engine.execute(query)
# Extract some values we need for building the table.
table_headers = results.keys()
table_body = results
max_cols = len(table_headers)
max_header_cols = max_cols
# Handle the width settings and title
col_widths = self.get_column_widths(max_cols)
title, messages = self.make_title()
# Build the node containing the table content

View Slide

line=self.lineno)
return [error]
)
# Run the query

View Slide

line=self.lineno)
return [error]
)
# Run the query

View Slide

# Run the query
table_node = self.build_table(table_body,
col_widths,
table_headers)
table_node['classes'] += self.options.get('class', [])
self.add_name(table_node)
if title:
table_node.insert(0, title)
return [table_node] + messages
def build_table(self, table_data, col_widths, headers):

View Slide

# Run the query
col_widths,
table_headers)
if title:

View Slide

# Run the query
col_widths,
table_headers)
if title:

View Slide

# Run the query
col_widths,
table_headers)
if title:

View Slide

col_widths,
table_headers)
if title:
table = nodes.table()
# Set up the column specifications
# based on the widths.
tgroup = nodes.tgroup(cols=len(col_widths))
table += tgroup
tgroup.extend(nodes.colspec(colwidth=col_width)
for col_width in col_widths)
# Set the headers
thead = nodes.thead()
tgroup += thead
row_node = nodes.row()

View Slide

col_widths,
table_headers)
if title:
table += tgroup
# Set the headers
tgroup += thead

View Slide

col_widths,
table_headers)
if title:
table += tgroup
# Set the headers
tgroup += thead

Table Node Hierarchy

table

tgroup
table

tgroup
colspec
table

tgroup
colspec thead
table

View Slide

tgroup
colspec thead
row
table

View Slide

tgroup
colspec thead
row
entry
table

View Slide

tgroup
colspec thead
row
entry
paragraph
table

View Slide

tgroup
colspec tbody
thead
row
entry
paragraph
table

View Slide

tgroup
colspec tbody
thead
row
entry
paragraph
row
entry
paragraph
table

View Slide

table += tgroup
# Set the headers
tgroup += thead
thead += row_node
row_node.extend(
nodes.entry(h, nodes.paragraph(text=h))
for h in headers
)
# The body of the table is made up of rows.
# Each row contains a series of entries,
# and each entry contains a paragraph of text.
tbody = nodes.tbody()
tgroup += tbody
tgroup
colspec
table

View Slide

table += tgroup
# Set the headers
tgroup += thead
thead += row_node
row_node.extend(
for h in headers
)
tgroup += tbody
tgroup
colspec
table

View Slide

table += tgroup
# Set the headers
tgroup += thead
thead += row_node
row_node.extend(
for h in headers
)
tgroup += tbody
tgroup
colspec
table

View Slide

table += tgroup
# Set the headers
tgroup += thead
thead += row_node
row_node.extend(
for h in headers
)
tgroup += tbody
tgroup
colspec
table

View Slide

# Set the headers
tgroup += thead
thead += row_node
row_node.extend(
for h in headers
)
tgroup += tbody
rows = []
for row in table_data:
trow = nodes.row()
for cell in row:
entry = nodes.entry()
para = nodes.paragraph(
text=unicode(cell))
entry += para
trow += entry
thead
row
entry
paragraph

View Slide

# Set the headers
tgroup += thead
thead += row_node
row_node.extend(
for h in headers
)
tgroup += tbody
rows = []
trow = nodes.row()
for cell in row:
text=unicode(cell))
entry += para
trow += entry
thead
row
entry
paragraph

View Slide

# Set the headers
tgroup += thead
thead += row_node
row_node.extend(
for h in headers
)
tgroup += tbody
rows = []
trow = nodes.row()
for cell in row:
text=unicode(cell))
entry += para
trow += entry
thead
row
entry
paragraph

View Slide

# Set the headers
tgroup += thead
thead += row_node
row_node.extend(
for h in headers
)
tgroup += tbody
rows = []
trow = nodes.row()
for cell in row:
text=unicode(cell))
entry += para
trow += entry
thead
row
entry
paragraph

View Slide

)
tgroup += tbody
rows = []
trow = nodes.row()
for cell in row:
text=unicode(cell))
entry += para
trow += entry
rows.append(trow)
tbody.extend(rows)
#print table
return table
def setup(app):
app.info('Initializing SQLTable')
app.add_config_value('sqltable_connection_string',
'',
'env')
tbody
row
entry
paragraph

View Slide

)
tgroup += tbody
rows = []
trow = nodes.row()
for cell in row:
text=unicode(cell))
entry += para
trow += entry
rows.append(trow)
tbody.extend(rows)
#print table
return table
def setup(app):
'',
'env')
tbody
row
entry
paragraph

View Slide

)
tgroup += tbody
rows = []
trow = nodes.row()
for cell in row:
text=unicode(cell))
entry += para
trow += entry
rows.append(trow)
tbody.extend(rows)
#print table
return table
def setup(app):
'',
'env')
tbody
row
entry
paragraph

View Slide

)
tgroup += tbody
rows = []
trow = nodes.row()
for cell in row:
text=unicode(cell))
entry += para
trow += entry
rows.append(trow)
tbody.extend(rows)
#print table
return table
def setup(app):
'',
'env')
tbody
row
entry
paragraph

View Slide

)
tgroup += tbody
rows = []
trow = nodes.row()
for cell in row:
text=unicode(cell))
entry += para
trow += entry
rows.append(trow)
tbody.extend(rows)
#print table
return table
def setup(app):
'',
'env')
tbody
row
entry
paragraph

View Slide

select
name as 'Name',
email as 'E-mail'
from
users
order by
Name asc

View Slide

select
name as 'Name',
email as 'E-mail'
from
users
order by
Name asc

View Slide

select
name as 'Name',
email as 'E-mail'
from
users
order by
Name asc

View Slide

select
name as 'Name',
email as 'E-mail'
from
users
order by
Name asc

View Slide

Build
Environment
docutils
parser
Sphinx
Builder
Sphinx
Application
.rst .pdf .html

.rst

.rst

.rst
word
list
check

.rst
word
list
check

section
title paragraph
#text
#text
#text

View Slide

section
title paragraph
#text
#text
#text

View Slide

from sphinx.builders import Builder
class SpellingBuilder(Builder):
name = 'spelling'
def init(self):
self.docnames = []
self.document_data = []
project_words = os.path.join(self.srcdir,
self.config.spelling_word_list_filename)
self.checker = SpellingChecker(
lang=self.config.spelling_lang,
suggest=self.config.spelling_show_suggestions,
word_list_filename=project_words,
)
self.output_filename = os.path.join(self.outdir,
'output.txt')
self.output = codecs.open(self.output_filename,
'wt', encoding='UTF-8')
def write_doc(self, docname, doctree):

View Slide

name = 'spelling'
def init(self):
self.docnames = []
)
'output.txt')

View Slide

name = 'spelling'
def init(self):
self.docnames = []
)
'output.txt')

View Slide

name = 'spelling'
def init(self):
self.docnames = []
)
'output.txt')

View Slide

name = 'spelling'
def init(self):
self.docnames = []
)
'output.txt')

View Slide

filename = self.env.doc2path(docname, base=None)
for node in doctree.traverse(docutils.nodes.Text):
if (node.tagname == '#text' and
node.parent and
node.parent.tagname in TEXT_NODES):
# Determine the line number for this node
lineno = get_line_number(node)
# Check the text of the node.
for word, suggestions in
self.checker.check(node.astext()):
msg_parts = [ docname ]
if lineno:
msg_parts.append(
darkgreen('(line %3d)' % lineno)
)
msg_parts.append(red(word))
msg_parts.append(
self.format_suggestions(suggestions)
)

View Slide

node.parent and
if lineno:
msg_parts.append(
)
msg_parts.append(
)

View Slide

node.parent and
if lineno:
msg_parts.append(
)
msg_parts.append(
)

View Slide

node.parent and
if lineno:
msg_parts.append(
)
msg_parts.append(
)

View Slide

if lineno:
msg_parts.append(
)
msg_parts.append(
)
msg = ' '.join(msg_parts)
self.info(msg)
self.output.write(u"%s:%s: (%s) %s\n" % (
self.env.doc2path(docname, None),
lineno, word,
self.format_suggestions(suggestions),
))
# Found at least one bad spelling
self.app.statuscode = 1
return
def finish(self):
self.output.close()
self.info('Output written to %s' % self.output_filename)
return

View Slide

if lineno:
msg_parts.append(
)
msg_parts.append(
)
msg = ' '.join(msg_parts)
self.info(msg)
self.output.write(u"%s:%s: (%s) %s\n" % (
lineno, word,
))
return
def finish(self):
self.output.close()
return

View Slide

lineno, word,
))
return
def finish(self):
self.output.close()
return
def setup(app):
app.info('Initializing Spelling Checker')
app.add_builder(SpellingBuilder)
return

View Slide

lineno, word,
))
return
def finish(self):
self.output.close()
return
def setup(app):
return

View Slide

lineno, word,
))
return
def finish(self):
self.output.close()
return
def setup(app):
return

View Slide

$ sphinx-build -b spelling -d build/doctrees \
source build/spelling
Running Sphinx v1.1.2
Initializing Spelling Checker
loading reprint directive
loading books directives
loading pickled environment... done
building [spelling]: all documents
updating environment: 0 added, 0 changed, 0 removed
looking for now-outdated files... none found
preparing documents... done
index (line 13) blarg ["blare", "blarney", "Blair", "blamer",
"blazer", "blog"]
writing output... [100%]
Spelling checker messages written to /Users/dhellmann/Devel/
website/website/build/spelling/output.txt
build finished with problems.

View Slide

"blazer", "blog"]

View Slide

"blazer", "blog"]

View Slide

Better Documentation Through Automation:
Creating docutils & Sphinx Extensions
Doug Hellmann
@doughellmann
PyCon 2013
http://docutils.sourceforge.net/
https://bitbucket.org/birkenfeld/sphinx-contrib/
http://packages.python.org/pyenchant/
http://doughellmann.com

View Slide

Better Documentation Through Automation: Creating docutils & Sphinx Extensions

Better Documentation Through Automation: Creating docutils & Sphinx Extensions

doughellmann

More Decks by doughellmann

Other Decks in Programming

Featured

Transcript