How we can solve DM’s Documentation ‘Problem’ Jonathan Sick @jonathansick LSST DM (SQuaRE) 

Principles of Stack Docs 1. Content is versioned with git 2. Static files → no content management system 3. Content lives with code 4. Continuously integrate docstrings/tutorials 5. Maintainable, mechanized content 6. Every stack developer is a doc writer 7. Aligned with the Python ecosystem 8. Be good to the reader 9. Don’t document broken code; fix it 2 

Let’s use Sphinx • Prose is reStructuredText - reST markup is extensible • Build to static HTML, LaTeX, etc. • Jinja2 templates, bring your own CSS/JavaScript • API Reference generation from docstrings - Python: autodoc/numpydoc - C++: Breathe bridges Doxygen XML • Sphinx is completely extensible (with Python) - Add reST roles (inline) and directives (blocks) 3 

Versioning Docs 4 

Versioning Docs 5 

Versioning Docs 6 dm.lsst.org/docs/v10.1/ Tagged releases get permanent URLs dm.lsst.org/docs/[git SHA1 or branch name]/ Jenkins CI builds make temporary docs 

Content Strategy 7 Getting Started Overview Install Working with packages Tutorials Processing a single image Processing with the Butler … Stack Packages afw meas_base … Developer Guide Development workflow Code Standards … overview.rst install.rst eups.rst github.com/lsst/afw github.com/lsst/meas_base dev/workflow.rst dev/standards.rst github.com/lsst/single_img_tutorial github.com/lsst/butler_tutorial reST files in docs repo docs live in pkg. repo we’ll provide doc templates for devs 

Write for the reader. 8 

Write for the reader. 9 

Write for the reader. 10 I want to install v10.1 ▾ on Mac OS X Yosemite ▾ from source ▾ . Install the Stack Get Dependencies Install Test I use zsh ▾ . 

Tutorials/Walk Throughs • Tutorials must teach realistic workflows and best practices; quick hacks not wanted. • Lightweight analysis tutorials - might be backed by Jupyter notebooks - rendered into documentation site • Pipeline development tutorials: - text + git repo - each tutorial stage is tagged 11 

Package Docs • Package docs live in the package’s repo 1. User guide / lightweight examples 2. API Reference (docstrings) • We will provide a clear template and guidelines 12 package_name Python API Reference C++ API Reference User guide + examples API with examples in docstrings 

C++ API Docs 13 doxygen I’m sorry. breathe bridges doxygen XML to Sphinx 

Docstring Formats: Sphinx 14 def foobar(a, b, keyword=None): """Fetches rows from a Bigtable. More paragraphs of documentation. :param a: something. :type a: int :param b: Some other thing with some type. :type b: str :param keyword: An optional variable, that has a much longer explanation than the other args. :type keyword: bool :returns: A dict that means something to you: {'Serak': ('Rigel VII', 'Preparer'), 'Zim': ('Irk', 'Invader'), 'Lrrr': ('Omicron Persei 8', 'Emperor')} More explanation. :rtype: dict :raises IOError: An error occurred accessing something. """ 

Docstring Formats: Google 15 def foobar(a, b, keyword=None): """Fetches rows from a Bigtable. More paragraphs of documentation. Args: a: An int for something. b: Some other thing with some type. keyword: An optional variable, that has a much longer explanation than the other args. Returns: A dict that means something to you: {'Serak': ('Rigel VII', 'Preparer'), 'Zim': ('Irk', 'Invader'), 'Lrrr': ('Omicron Persei 8', 'Emperor')} More explanation. Raises: IOError: An error occurred accessing something. """ with napoleon http://bit.ly/1MBtOwW 

Docstring Formats: Numpy 16 def foobar(a, b, keyword=None): """Fetches rows from a Bigtable. More paragraphs of documentation. Parameters —————————— a : int An int for something. b : str Some other thing. keyword : str, optional An optional variable, that has a much longer explanation than the other args. Returns ——————— my_dict : dict Something that means something to you:: {'Serak': ('Rigel VII', 'Preparer'), 'Zim': ('Irk', 'Invader'), 'Lrrr': ('Omicron Persei 8', 'Emperor')} Raises —————— IOError: An error occurred accessing something. """ napoleon/numpydoc http://bit.ly/1LjBt2O 

DRY Docstrings 17 Can C++ docstrings percolate into the Python layer? C++ Only C++ & Python Python Only Need to balance developer efficiency, C++ and Python doc parity, and user experience. Interactive python docstrings matter. 

Beyond the minimum viable doc • Responsive doc site - Sass / Susy / gulp / responsive [images] - CI of website - Optimize navigation for long pages • Templates for docs and tutorials • Style guide for docs / writing • Continuous integration of examples, tutorials • Integration with community.lsst.org • Academic citation guidance throughout • Even better search of code+docs (Elasticsearch?) 18 

Culture of Doc Salesforce (Heroku) has an API Design Review Board Solve API issues before shipping Doc & QA Teams are board members 1. Developers present API and fill out the documentation template 2. Achieve consensus over 1. API naming consistency 2. Design idioms/consistency 3. Error codes 4. Usability / documentability 19 Docs as QA