Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Skyfield and 15 Years of Bad APIs (Brandon Rhodes)

Skyfield and 15 Years of Bad APIs (Brandon Rhodes)

PyCon Canada

August 14, 2013
Tweet

More Decks by PyCon Canada

Other Decks in Education

Transcript

  1. Skyfield and 15 Years of Bad APIs
    @brandon_rhodes
    PyCon Canada
    August 2013

    View full-size slide

  2. Goal
    To reflect upon the practice of
    Python API design through my recent
    work on the Skyfield astronomy library

    View full-size slide

  3. Elwood Charles Downey et al
    1990 Ephem
    1993 XEphem

    View full-size slide

  4. e·phem·er·is — A table
    giving the coordinates of
    a celestial body at a number
    of specific times

    View full-size slide

  5. C
    D
    T 1
    9
    :
    0
    0
    :
    0
    0 4
    /
    3
    0
    /
    1
    9
    9
    0 | L
    S
    T 8
    :
    1
    9
    :
    5
    0 |
    U
    T
    C 0
    :
    0
    0
    :
    0
    0 5
    /
    0
    1
    /
    1
    9
    9
    0 | |
    J
    u
    l
    i
    a
    n
    D
    a
    t 2
    4
    4
    8
    0
    1
    2
    .
    5
    0
    0
    0
    0 | D
    a
    w
    n 4
    :
    1
    0 |
    W
    a
    t
    c
    h | D
    u
    s
    k 2
    2
    :
    1
    5 |
    L
    i
    s
    t
    i
    n
    g o
    f
    f | N
    i
    t
    e
    L
    n 5
    :
    5
    5 |
    P
    l
    o
    t o
    f
    f | N
    S
    t
    e
    p 1 |
    M
    e
    n
    u P
    l
    a
    n
    e
    t D
    a
    t
    a | S
    t
    p
    S
    z R
    T C
    L
    O
    C
    K |
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    -
    O
    C
    X R
    .
    A
    . D
    e
    c A
    z A
    l
    t H L
    o
    n
    g H L
    a
    t
    S
    u 2
    :
    3
    2
    .
    3 1
    4
    :
    5
    8 2
    7
    8
    :
    4
    0 1
    2
    :
    3
    8 2
    2
    0
    :
    2
    2
    M
    o 8
    :
    0
    9
    .
    9 2
    1
    :
    1
    1 1
    8
    6
    :
    0
    6 6
    5
    :
    5
    3 1
    1
    9
    :
    5
    5 1
    :
    0
    4
    M
    e 2
    :
    4
    9
    .
    4 1
    7
    :
    3
    9 2
    7
    7
    :
    4
    8 1
    7
    :
    2
    6 2
    1
    4
    :
    0
    8 1
    :
    4
    3
    V
    e 2
    3
    :
    4
    9
    .
    4 -
    2
    :
    2
    5 2
    9
    6
    :
    5
    3 -
    2
    7
    :
    3
    9 2
    8
    2
    :
    3
    9 -
    1
    :
    3
    0
    M
    a 2
    2
    :
    3
    9
    .
    8 -
    1
    0
    :
    0
    9 3
    0
    8
    :
    1
    7 -
    4
    4
    :
    1
    4 2
    9
    7
    :
    5
    6 -
    1
    :
    4
    3
    J
    u 6
    :
    3
    0
    .
    9 2
    3
    :
    2
    3 2
    3
    5
    :
    1
    3 5
    9
    :
    0
    4 1
    0
    6
    :
    1
    6 0
    :
    0
    8
    S
    a 1
    9
    :
    4
    9
    .
    6 -
    2
    0
    :
    5
    3 1
    7
    :
    2
    4 -
    6
    5
    :
    1
    4 2
    8
    9
    :
    4
    5 0
    :
    1
    0

    View full-size slide

  6. ©
    XEphem’s author reserves
    the right to distribute binaries,
    but allows free download of the source

    View full-size slide

  7. Missing header (.h)
    X
    1
    1
    /
    I
    n
    t
    r
    i
    n
    s
    i
    c
    .
    h
    : N
    o s
    u
    c
    h f
    i
    l
    e o
    r d
    i
    r
    e
    c
    t
    o
    r
    y
    .
    .
    .
    $ a
    p
    t
    -
    f
    i
    l
    e s
    e
    a
    r
    c
    h X
    1
    1
    /
    I
    n
    t
    r
    i
    n
    s
    i
    c
    .
    h
    l
    i
    b
    x
    t
    -
    d
    e
    v
    : /
    u
    s
    r
    /
    i
    n
    c
    l
    u
    d
    e
    /
    X
    1
    1
    /
    I
    n
    t
    r
    i
    n
    s
    i
    c
    .
    h
    Missing library (-l)
    /
    u
    s
    r
    /
    b
    i
    n
    /
    l
    d
    : c
    a
    n
    n
    o
    t f
    i
    n
    d -
    l
    X
    e
    x
    t
    .
    .
    .
    $ a
    p
    t
    -
    f
    i
    l
    e s
    e
    a
    r
    c
    h l
    i
    b
    X
    e
    x
    t
    .
    a
    l
    i
    b
    x
    e
    x
    t
    -
    d
    e
    v
    : /
    u
    s
    r
    /
    l
    i
    b
    /
    i
    3
    8
    6
    -
    l
    i
    n
    u
    x
    -
    g
    n
    u
    /
    l
    i
    b
    X
    e
    x
    t
    .
    a

    View full-size slide

  8. Instead of using a GUI,
    I wanted to write scripts

    View full-size slide

  9. Inside XEphem’s C-language
    source code is a computation
    engine called libastro

    View full-size slide

  10. XEphem PyEphem
    libastro → libastro

    View full-size slide

  11. “You have my full permission
    to go with what you have.”
    — Elwood’s generous reply!

    View full-size slide

  12. PyEphem = C code
    Wrapper around libastro

    View full-size slide

  13. 1998
    Beazley’s Simple Wrapper
    Interface Generator (SWIG)

    View full-size slide

  14. SWIG
    Exposed awkward details of C structs
    b
    o
    d
    y = O
    b
    j
    (
    )
    b
    o
    d
    y
    .
    a
    n
    y
    .
    t
    y
    p
    e = e
    p
    h
    e
    m
    .
    P
    L
    A
    N
    E
    T
    b
    o
    d
    y
    .
    p
    l
    .
    c
    o
    d
    e = e
    p
    h
    e
    m
    .
    S
    U
    N
    e
    p
    h
    e
    m
    .
    c
    o
    m
    p
    u
    t
    e
    L
    o
    c
    a
    t
    i
    o
    n
    (
    c
    i
    r
    c
    u
    m
    , b
    o
    d
    y
    )
    p
    r
    i
    n
    t e
    p
    h
    e
    m
    .
    f
    o
    r
    m
    a
    t
    H
    o
    u
    r
    s
    (
    o
    .
    a
    n
    y
    .
    r
    a
    , 3
    6
    0
    0
    0
    )
    p
    r
    i
    n
    t e
    p
    h
    e
    m
    .
    f
    o
    r
    m
    a
    t
    D
    e
    g
    r
    e
    e
    s
    (
    o
    .
    a
    n
    y
    .
    d
    e
    c
    , 3
    6
    0
    0
    )

    View full-size slide

  15. 2003
    Hand-written C that uses
    Python 2.2 superpowers

    View full-size slide

  16. Python 2.2 made attribute
    access easy to customize
    s
    t
    a
    t
    i
    c P
    y
    G
    e
    t
    S
    e
    t
    D
    e
    f b
    o
    d
    y
    _
    g
    e
    t
    s
    e
    t
    [
    ] = {
    {
    "
    r
    a
    "
    , g
    e
    t
    _
    r
    a
    , 0
    , "
    r
    i
    g
    h
    t a
    s
    c
    e
    n
    s
    i
    o
    n
    "
    }
    ,
    {
    "
    d
    e
    c
    "
    , g
    e
    t
    _
    d
    e
    c
    , 0
    , "
    d
    e
    c
    l
    i
    n
    a
    t
    i
    o
    n
    "
    }
    ,
    {
    "
    e
    l
    o
    n
    g
    "
    , g
    e
    t
    _
    e
    l
    o
    n
    g
    , 0
    , "
    e
    l
    o
    n
    g
    a
    t
    i
    o
    n
    "
    }
    ,
    {
    "
    m
    a
    g
    "
    , g
    e
    t
    _
    m
    a
    g
    , 0
    , "
    m
    a
    g
    n
    i
    t
    u
    d
    e
    "
    }
    ,

    }

    View full-size slide

  17. And, I added a pure-Python wrapper
    on top like h
    a
    s
    h
    l
    i
    b
    , s
    q
    l
    i
    t
    e
    3
    , and s
    s
    l
    # e
    p
    h
    e
    m
    /
    _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y
    i
    m
    p
    o
    r
    t _
    l
    i
    b
    a
    s
    t
    r
    o

    View full-size slide

  18. More Pythonic
    edition of PyEphem
    m
    a
    r
    s = e
    p
    h
    e
    m
    .
    M
    a
    r
    s
    (
    )
    m
    a
    r
    s
    .
    c
    o
    m
    p
    u
    t
    e
    (
    )
    p
    r
    i
    n
    t m
    a
    r
    s
    .
    r
    a
    , m
    a
    r
    s
    .
    d
    e
    c

    View full-size slide

  19. But, both interfaces
    were still based on C code
    and required compilation

    View full-size slide

  20. S
    u
    b
    j
    e
    c
    t
    : P
    y
    E
    p
    h
    e
    m W
    i
    n
    3
    2 b
    u
    i
    l
    d e
    r
    r
    o
    r
    s
    S
    u
    b
    j
    e
    c
    t
    : w
    i
    n
    3
    2
    , d
    o
    e
    s P
    h
    E
    p
    h
    e
    m w
    o
    r
    k t
    h
    e
    r
    e t
    o
    o
    ?
    S
    u
    b
    j
    e
    c
    t
    : t
    r
    y
    i
    n
    g t
    o d
    o
    w
    n
    l
    o
    a
    d b
    u
    t I c
    a
    n
    t u
    n
    z
    i
    p i
    t

    View full-size slide

  21. S
    u
    b
    j
    e
    c
    t
    : p
    y
    e
    p
    h
    e
    m o
    n M
    a
    c P
    P
    C
    S
    u
    b
    j
    e
    c
    t
    : p
    y
    E
    p
    h
    e
    m w
    o
    n

    t b
    u
    i
    l
    d o
    n S
    n
    o
    w L
    e
    o
    p
    a
    r
    d
    S
    u
    b
    j
    e
    c
    t
    : P
    y
    E
    p
    h
    e
    m … I
    n
    s
    t
    a
    l
    l
    a
    t
    i
    o
    n e
    r
    r
    o
    r i
    n o
    p
    e
    n
    s
    u
    s
    e
    S
    u
    b
    j
    e
    c
    t
    : P
    y
    E
    p
    h
    e
    m o
    n U
    b
    u
    n
    t
    u 1
    0
    .
    1
    0
    S
    u
    b
    j
    e
    c
    t
    : P
    y
    e
    p
    h
    e
    m o
    n a 6
    4
    -
    b
    i
    t W
    i
    n 7 P
    C
    ?

    View full-size slide

  22. Mac
    Sometimes a problem,
    but now on MacPorts!
    Windows
    p
    y
    t
    h
    o
    n s
    e
    t
    u
    p
    .
    p
    y b
    d
    i
    s
    t
    _
    w
    i
    n
    i
    n
    s
    t
    mingw — Open, but quirky
    Visual Studio Express — Works!

    View full-size slide

  23. C extensions
    They are difficult to—
    Install
    Distribute
    Maintain

    View full-size slide

  24. I slowly grew open to the idea
    of an alternative approach

    View full-size slide

  25. An email
    “I’m interested in ephemeris options
    for astrology apps. You probably know
    about the Swiss Ephemeris. Do you know
    how it compares in accuracy?”

    View full-size slide

  26. libastro
    Predicts planetary
    positions using VSOP87
    1987

    View full-size slide

  27. Swiss Ephemeris
    “based upon the DE406,”
    the JPL Long Ephemeris
    1997

    View full-size slide

  28. But wait!
    # f
    t
    p
    :
    /
    /
    s
    s
    d
    .
    j
    p
    l
    .
    n
    a
    s
    a
    .
    g
    o
    v
    /
    p
    u
    b
    /
    e
    p
    h
    /
    p
    l
    a
    n
    e
    t
    s
    /
    a
    s
    c
    i
    i
    /
    N
    a
    m
    e D
    a
    t
    e M
    o
    d
    i
    f
    i
    e
    d

    d
    e
    4
    0
    5
    / 1
    0
    /
    7
    /
    0
    7 8
    :
    0
    0
    :
    0
    0 P
    M
    d
    e
    4
    0
    6
    / 3
    /
    2
    2
    /
    1
    1 8
    :
    0
    0
    :
    0
    0 P
    M

    d
    e
    4
    2
    1
    / 2
    /
    6
    /
    1
    3 7
    :
    0
    0
    :
    0
    0 P
    M
    d
    e
    4
    2
    2
    / 8
    /
    3
    /
    1
    1 8
    :
    0
    0
    :
    0
    0 P
    M
    d
    e
    4
    2
    3
    / 3
    /
    3
    0
    /
    1
    0 8
    :
    0
    0
    :
    0
    0 P
    M

    View full-size slide

  29. DE421
    More recent
    More accurate
    “planetary navigation accuracies”

    View full-size slide

  30. Wait—what?
    DE421 uses the
    “International Celestial Reference Frame”

    View full-size slide

  31. USNO Circular 179
    George H. Kaplan
    2005

    View full-size slide

  32. “The … resolutions passed by
    the International Astronomical Union
    at its General Assemblies in 1997 and 2000
    are the most significant set of international
    agreements in positional astronomy in
    several decades and arguably since
    the Paris conference of 1896.”

    View full-size slide

  33. Time for a rewrite!
    Not a simple re-implementation
    But writing, in Python, a complete
    replacement of the old algorithms
    used throughout PyEphem

    View full-size slide

  34. Goals
    Replace entire API
    Integrate with scientific Python
    Support vector NumPy computations
    Avoid re-inventing SciPy wheels

    View full-size slide

  35. Example: Sunrise
    PyEphem — custom hand-written
    version of Newton’s Method
    New approach — an IPython Notebook that
    finds sunrise with s
    c
    i
    p
    y
    .
    o
    p
    t
    i
    m
    i
    z
    e

    View full-size slide

  36. Tools
    git
    Jedi
    py.test
    tox

    View full-size slide

  37. py.test
    p
    y
    .
    t
    e
    s
    t -
    -
    p
    y
    a
    r
    g
    s s
    k
    y
    f
    i
    e
    l
    d \
    -
    -
    d
    o
    c
    t
    e
    s
    t
    -
    g
    l
    o
    b
    =
    '
    *
    .
    r
    s
    t
    '

    View full-size slide

  38. py.test
    a
    s
    s
    e
    r
    t c
    .
    c
    a
    l
    _
    d
    a
    t
    e
    (
    j
    d
    ) =
    = t
    i
    m
    e
    s
    c
    a
    l
    e
    s
    .
    c
    a
    l
    _
    d
    a
    t
    e
    (
    j
    d
    )

    View full-size slide

  39. I distribute tests
    s
    e
    t
    u
    p
    (
    .
    .
    .
    p
    a
    c
    k
    a
    g
    e
    s
    =
    [
    '
    s
    k
    y
    f
    i
    e
    l
    d
    '
    , '
    s
    k
    y
    f
    i
    e
    l
    d
    .
    t
    e
    s
    t
    s
    '
    ]
    ,
    .
    .
    .
    )

    View full-size slide

  40. I distribute docs
    s
    e
    t
    u
    p
    (
    .
    .
    .
    p
    a
    c
    k
    a
    g
    e
    _
    d
    a
    t
    a = {
    '
    s
    k
    y
    f
    i
    e
    l
    d
    '
    : [
    '
    d
    o
    c
    u
    m
    e
    n
    t
    a
    t
    i
    o
    n
    /
    *
    .
    r
    s
    t
    '
    ]
    }
    ,
    .
    .
    .
    )

    View full-size slide

  41. GPL
    F
    r
    e
    e w
    o
    r
    l
    d C
    l
    o
    s
    e
    d w
    o
    r
    l
    d









    ─ ─











    A
    w
    e
    s
    o
    m
    e
    ! L
    e
    s
    s a
    w
    e
    s
    o
    m
    e
    ↑ ↑
    Y
    o
    u
    r l
    i
    b
    r
    a
    r
    y → × A
    l
    t
    e
    r
    n
    a
    t
    i
    v
    e
    ?
    R
    e
    w
    r
    i
    t
    e
    ?

    View full-size slide

  42. MIT/BSD
    F
    r
    e
    e w
    o
    r
    l
    d C
    l
    o
    s
    e
    d w
    o
    r
    l
    d









    ─ ─











    A
    w
    e
    s
    o
    m
    e
    ! C
    l
    o
    s
    e
    d a
    w
    e
    s
    o
    m
    e
    ↑ ↑
    Y
    o
    u
    r l
    i
    b
    r
    a
    r
    y → Y
    o
    u
    r l
    i
    b
    r
    a
    r
    y

    View full-size slide

  43. (Look closely — Open Source everywhere!)
    F
    r
    e
    e w
    o
    r
    l
    d C
    l
    o
    s
    e
    d w
    o
    r
    l
    d









    ─ ─











    A
    w
    e
    s
    o
    m
    e
    ! C
    l
    o
    s
    e
    d a
    w
    e
    s
    o
    m
    e
    ↑ ↑
    Y
    o
    u
    r l
    i
    b
    r
    a
    r
    y → Y
    o
    u
    r l
    i
    b
    r
    a
    r
    y

    View full-size slide

  44. This is Python’s own model,
    and Python is increasingly everywhere

    View full-size slide

  45. MIT/BSD
    The free kind of freedom is what wins

    View full-size slide

  46. single code base
    that runs on both
    Python 2 and Python 3

    View full-size slide

  47. Release small projects fast
    j
    p
    l
    e
    p
    h
    e
    m
    s
    g
    p
    4

    View full-size slide

  48. Carefully monitor my emotions

    View full-size slide

  49. Emotions tell me whether
    I am programming right

    View full-size slide

  50. Frustration?
    Clenching teeth?
    Trapped?

    View full-size slide

  51. Then I might be doing it wrong

    View full-size slide

  52. Frustration often signals
    inadequate project structure

    View full-size slide

  53. I notice that tiny goals
    are easier to reach and
    keep me calm

    View full-size slide

  54. “Test-driven development is a way of
    managing fear during programming.”
    — Kent Beck,
    Test Driven Development By Example

    View full-size slide

  55. Version Control
    “How soon can I commit
    so that I can’t lose what
    I’ve just typed?”

    View full-size slide

  56. Young
    “How can I split this task into
    smaller pieces for the computer?”
    Old
    “How can I split this task into
    smaller pieces for me?”

    View full-size slide

  57. Big slog: risks everything!
    here →→→→→ new feature
    here → • → • → • → • → new feature
    Incremental: cheap to revert

    View full-size slide

  58. Always use g
    i
    t s
    t
    a
    s
    h
    to revert and try again

    View full-size slide

  59. Aim for a Moon Shot
    Write up an end-to-end example
    then move straight there

    View full-size slide

  60. The sins of my APIs

    View full-size slide

  61. Sin: Inscrutable names
    “Explicit is better than implicit”
    d = '
    2
    0
    1
    2
    /
    1
    1
    /
    9
    '
    m = e
    p
    h
    e
    m
    .
    m
    a
    r
    s
    (
    d
    )
    p
    r
    i
    n
    t
    (
    m
    .
    a
    _
    r
    a
    , m
    .
    a
    _
    d
    e
    c
    )
    # S
    k
    y
    f
    i
    e
    l
    d
    , i
    n
    s
    t
    e
    a
    d
    :
    d = J
    u
    l
    i
    a
    n
    D
    a
    t
    e
    (
    '
    2
    0
    1
    2
    /
    1
    1
    /
    9
    '
    )
    p = e
    a
    r
    t
    h
    (
    d
    )
    .
    o
    b
    s
    e
    r
    v
    e
    (
    m
    a
    r
    s
    )
    .
    a
    s
    t
    r
    o
    m
    e
    t
    r
    i
    c
    (
    )
    p
    r
    i
    n
    t
    (
    p
    .
    r
    a
    , p
    .
    d
    e
    c
    )

    View full-size slide

  62. Sin: storing results on object
    m
    a
    r
    s = e
    p
    h
    e
    m
    .
    M
    a
    r
    s
    (
    )
    m
    a
    r
    s
    .
    c
    o
    m
    p
    u
    t
    e
    (
    '
    2
    0
    1
    2
    /
    1
    1
    /
    9
    '
    )
    p
    r
    i
    n
    t
    (
    m
    .
    r
    a
    , m
    .
    d
    e
    c
    )
    # M
    a
    r
    s
    # .
    n
    a
    m
    e
    # .
    d
    a
    t
    e
    # .
    c
    o
    m
    p
    u
    t
    e
    (
    ) →

    # .
    r
    a ↖ ↓
    # .
    d
    e
    c ←







    # ⋮

    View full-size slide

  63. # P
    y
    E
    p
    h
    e
    m
    :
    p
    o
    s
    i
    t
    i
    o
    n
    s = [
    ]
    f
    o
    r d
    a
    t
    e i
    n d
    a
    t
    e
    s
    :
    m
    a
    r
    s
    .
    c
    o
    m
    p
    u
    t
    e
    (
    d
    )
    p
    o
    s
    i
    t
    i
    o
    n
    s
    .
    a
    p
    p
    e
    n
    d
    (
    (
    m
    a
    r
    s
    .
    r
    a
    , m
    a
    r
    s
    .
    d
    e
    c
    )
    )
    # S
    k
    y
    f
    i
    e
    l
    d
    :
    c
    o
    o
    r
    d
    s = [
    m
    a
    r
    s
    (
    d
    )
    .
    a
    s
    t
    r
    o
    m
    e
    t
    r
    i
    c
    (
    ) f
    o
    r d i
    n d
    a
    t
    e
    s
    ]
    p
    o
    s
    i
    t
    i
    o
    n
    s = [
    (
    c
    .
    r
    a
    , c
    .
    d
    e
    c
    ) f
    o
    r c i
    n c
    o
    o
    r
    d
    s
    ]

    View full-size slide

  64. Look familiar?

    View full-size slide

  65. l
    e
    t
    t
    e
    r
    s = l
    i
    s
    t
    (
    s
    e
    t
    (
    m
    e
    s
    s
    a
    g
    e
    _
    s
    t
    r
    i
    n
    g
    )
    )
    l
    e
    t
    t
    e
    r
    s
    .
    s
    o
    r
    t
    (
    )
    p
    r
    i
    n
    t '
    '
    .
    j
    o
    i
    n
    (
    l
    e
    t
    t
    e
    r
    s
    )
    # v
    s
    p
    r
    i
    n
    t '
    '
    .
    j
    o
    i
    n
    (
    s
    o
    r
    t
    e
    d
    (
    s
    e
    t
    (
    m
    e
    s
    s
    a
    g
    e
    _
    s
    t
    r
    i
    n
    g
    )
    )
    )

    View full-size slide

  66. o
    u
    t
    p
    u
    t
    s = s
    o
    r
    t
    e
    d
    (
    i
    n
    p
    u
    t
    s
    )
    Making code more functional
    is a big part of Pythonic

    View full-size slide

  67. Sin: Concealing expense
    Your API is the only lifeline
    the programmer has to managing
    complexity and expense!

    View full-size slide

  68. # P
    y
    E
    p
    h
    e
    m
    m = e
    p
    h
    e
    m
    .
    m
    a
    r
    s
    (
    '
    2
    0
    1
    2
    /
    1
    1
    /
    9
    '
    )
    p
    r
    i
    n
    t
    (
    m
    .
    n
    a
    m
    e
    ) # z
    e
    r
    o w
    o
    r
    k
    p
    r
    i
    n
    t
    (
    m
    .
    r
    a
    , m
    .
    d
    e
    c
    ) # c
    o
    m
    p
    u
    t
    e
    d
    p
    r
    i
    n
    t
    (
    m
    .
    r
    i
    s
    e
    _
    t
    i
    m
    e
    ) # e
    x
    p
    e
    n
    s
    i
    v
    e
    !

    View full-size slide

  69. Guideline
    It’s okay to hide quick
    conveniences behind properties
    But expensive operations
    should always look like calls!

    View full-size slide

  70. m
    a
    r
    s
    .
    n
    a
    m
    e # l
    o
    o
    k
    s c
    h
    e
    a
    p
    m
    a
    r
    s
    .
    a
    p
    p
    a
    r
    e
    n
    t
    (
    ) # l
    o
    o
    k
    s e
    x
    p
    e
    n
    s
    i
    v
    e
    !
    Use this difference to
    train your customers!

    View full-size slide

  71. # “
    I c
    a
    n u
    s
    e t
    h
    i
    s o
    v
    e
    r a
    n
    d o
    v
    e
    r a
    g
    a
    i
    n
    !

    m
    a
    r
    s
    .
    n
    a
    m
    e
    # “
    L
    o
    o
    k
    s e
    x
    p
    e
    n
    s
    i
    v
    e
    ; I

    l
    l s
    a
    v
    e t
    h
    e
    # r
    e
    s
    u
    l
    t t
    o a l
    o
    c
    a
    l n
    a
    m
    e i
    n
    s
    t
    e
    a
    d
    .

    m
    a
    r
    s
    .
    a
    p
    p
    a
    r
    e
    n
    t
    (
    )

    View full-size slide

  72. The big lesson?

    View full-size slide

  73. Write APIs that TEACH!

    View full-size slide

  74. Write APIs that teach
    You know how your API works
    and how it can best be used

    View full-size slide

  75. Share as much of that
    knowledge as possible

    View full-size slide

  76. Write APIs that teach
    While an API should hide complexity
    it should also suggest best practices

    View full-size slide

  77. Write APIs that teach
    r = u
    r
    l
    l
    i
    b
    2
    .
    u
    r
    l
    o
    p
    e
    n
    (
    u
    r
    l
    ) # b
    a
    d
    r = r
    e
    q
    u
    e
    s
    t
    s
    .
    g
    e
    t
    (
    u
    r
    l
    ) # g
    o
    o
    d

    View full-size slide

  78. Why?
    r = r
    e
    q
    u
    e
    s
    t
    s
    .
    g
    e
    t
    (
    u
    r
    l
    )
    The very first line of r
    e
    q
    u
    e
    s
    t
    s code
    teaches users their first HTTP verb —
    without their even knowing it

    View full-size slide

  79. PyEphem made every coordinate
    an attribute — looking exactly the same
    with no visible relationship!
    m
    a
    r
    s
    .
    a
    _
    r
    a
    , m
    a
    r
    s
    .
    a
    _
    d
    e
    c # a
    s
    t
    r
    o
    m
    e
    t
    r
    i
    c
    m
    a
    r
    s
    .
    g
    _
    r
    a
    , m
    a
    r
    s
    .
    g
    _
    d
    e
    c # a
    p
    p
    a
    r
    e
    n
    t g
    e
    o
    c
    e
    n
    t
    r
    i
    c
    m
    a
    r
    s
    .
    r
    a
    , m
    a
    r
    s
    .
    d
    e
    c # a
    p
    p
    a
    r
    e
    n
    t t
    o
    p
    o
    c
    e
    n
    t
    r
    i
    c
    m
    a
    r
    s
    .
    a
    l
    t
    , m
    a
    r
    s
    .
    a
    z # a
    p
    p
    a
    r
    e
    n
    t h
    o
    r
    i
    z
    o
    n
    t
    a
    l

    View full-size slide

  80. Skyfield has you to build a
    result from smaller operations
    h
    e
    r
    e = t
    o
    r
    o
    n
    t
    o
    (
    j
    d
    )
    h
    e
    r
    e
    .
    o
    b
    s
    e
    r
    v
    e
    (
    m
    a
    r
    s
    )
    .
    a
    s
    t
    r
    o
    m
    e
    t
    r
    i
    c
    (
    )
    h
    e
    r
    e
    .
    o
    b
    s
    e
    r
    v
    e
    (
    m
    a
    r
    s
    )
    .
    a
    p
    p
    a
    r
    e
    n
    t
    (
    )
    .
    e
    q
    u
    a
    t
    o
    r
    i
    a
    l
    (
    )
    h
    e
    r
    e
    .
    o
    b
    s
    e
    r
    v
    e
    (
    m
    a
    r
    s
    )
    .
    a
    p
    p
    a
    r
    e
    n
    t
    (
    )
    .
    h
    o
    r
    i
    z
    o
    n
    t
    a
    l
    (
    )

    View full-size slide

  81. Sin: Confusing functions and methods
    Python support both procedural
    and object-based methods, but
    choosing can be difficult

    View full-size slide

  82. PyEphem
    m = M
    a
    r
    s
    (
    d
    a
    t
    e
    )
    p
    r
    i
    n
    t
    (
    c
    o
    n
    s
    t
    e
    l
    l
    a
    t
    i
    o
    n
    (
    m
    .
    r
    a
    , m
    .
    d
    e
    c
    )
    )
    # T
    h
    i
    s f
    u
    n
    c
    t
    i
    o
    n c
    a
    n O
    N
    L
    Y E
    V
    E
    R b
    e p
    a
    s
    s
    e
    d
    # a r
    i
    g
    h
    t a
    s
    c
    e
    n
    s
    i
    o
    n a
    n
    d d
    e
    c
    l
    i
    n
    a
    t
    i
    o
    n
    ; w
    h
    y
    # n
    o
    t m
    a
    k
    e i
    t a c
    o
    o
    r
    d
    i
    n
    a
    t
    e m
    e
    t
    h
    o
    d
    ?

    View full-size slide

  83. So here are some guidelines
    that I have been using lately

    View full-size slide

  84. 1. Methods constitute Python’s
    built-in typecheck — the one clean
    way to limit the types that a function
    will accept as an argument

    View full-size slide

  85. If you are tempted to start a function
    with i
    f i
    s
    i
    n
    s
    t
    a
    n
    c
    e
    (

    ) then you
    might want a method instead!

    View full-size slide

  86. 2. f
    (
    x
    ) should by definition touch
    only public features of an x

    View full-size slide

  87. If f
    (
    x
    ) needs internal details,
    then either make it a method itself,
    or create a method that does the
    internal manipulation for it

    View full-size slide

  88. 3. f
    (
    x
    ) should by definition not
    mutate the state of x
    (If you need to mutate state from
    outside, try the Adapter Pattern!)

    View full-size slide

  89. PyEphem
    m = M
    a
    r
    s
    (
    d
    a
    t
    e
    )
    p
    r
    i
    n
    t m
    .
    r
    a # p
    r
    i
    n
    t
    s o
    n
    e t
    h
    i
    n
    g
    t
    o
    r
    o
    n
    t
    o
    .
    n
    e
    x
    t
    _
    r
    i
    s
    i
    n
    g
    (
    m
    )
    p
    r
    i
    n
    t m
    .
    r
    a # s
    o
    m
    e
    t
    h
    i
    n
    g d
    i
    f
    f
    e
    r
    e
    n
    t
    !

    View full-size slide

  90. 4. Methods are discoverable
    (Jedi!)

    View full-size slide

  91. Skyfield: New tricks

    View full-size slide

  92. What about when you do need
    to dynamically compute
    an attribute?

    View full-size slide

  93. c
    l
    a
    s
    s S
    a
    m
    p
    l
    e
    (
    o
    b
    j
    e
    c
    t
    )
    :
    @
    p
    r
    o
    p
    e
    r
    t
    y
    d
    e
    f l
    o
    u
    d
    (
    s
    e
    l
    f
    )
    :
    r
    e
    t
    u
    r
    n s
    e
    l
    f
    .
    m
    e
    s
    s
    a
    g
    e
    .
    u
    p
    p
    e
    r
    (
    )

    View full-size slide

  94. Problem: hidden expense
    So, we cache

    View full-size slide

  95. c
    l
    a
    s
    s S
    a
    m
    p
    l
    e
    (
    o
    b
    j
    e
    c
    t
    )
    :
    _
    l
    o
    u
    d = N
    o
    n
    e
    @
    p
    r
    o
    p
    e
    r
    t
    y
    d
    e
    f l
    o
    u
    d
    (
    s
    e
    l
    f
    )
    :
    i
    f _
    l
    o
    u
    d i
    s N
    o
    n
    e
    :
    s
    e
    l
    f
    .
    _
    l
    o
    u
    d = s
    e
    l
    f
    .
    m
    e
    s
    s
    a
    g
    e
    .
    u
    p
    p
    e
    r
    (
    )
    r
    e
    t
    u
    r
    n s
    e
    l
    f
    .
    _
    l
    o
    u
    d

    View full-size slide

  96. Problem: makes every
    access more expensive

    View full-size slide

  97. Solution: dunder-getattr!
    c
    l
    a
    s
    s S
    a
    m
    p
    l
    e
    (
    o
    b
    j
    e
    c
    t
    )
    :
    d
    e
    f _
    _
    g
    e
    t
    a
    t
    t
    r
    _
    _
    (
    s
    e
    l
    f
    , n
    a
    m
    e
    )
    :
    i
    f n
    a
    m
    e =
    = '
    l
    o
    u
    d
    '
    :
    s
    e
    l
    f
    .
    l
    o
    u
    d = s
    e
    l
    f
    .
    m
    e
    s
    s
    a
    g
    e
    .
    u
    p
    p
    e
    r
    (
    )
    r
    e
    t
    u
    r
    n s
    e
    l
    f
    .
    l
    o
    u
    d
    r
    a
    i
    s
    e A
    t
    t
    r
    i
    b
    u
    t
    e
    E
    r
    r
    o
    r
    (
    )

    View full-size slide

  98. Solution: dunder-getattr
    For a value frequently
    accessed in heavy numeric
    work, _
    _
    g
    e
    t
    a
    t
    t
    r
    _
    _
    (
    ) runs
    only on the first lookup!

    View full-size slide

  99. Trick: scalar style
    NumPy initially excited me
    by letting me write scalar code
    that also works on arrays!

    View full-size slide

  100. d
    e
    f f
    (
    x
    , y
    )
    :
    r
    e
    t
    u
    r
    n s
    q
    r
    t
    (
    x
    *
    x + y
    *
    y
    )
    p
    r
    i
    n
    t
    (
    f
    (
    3
    , 4
    )
    )
    # =
    > 5
    x = a
    r
    r
    a
    y
    (
    [
    3
    , 8
    , 6
    0
    ]
    )
    y = a
    r
    r
    a
    y
    (
    [
    4
    , 6
    , 8
    0
    ]
    )
    p
    r
    i
    n
    t
    (
    f
    (
    x
    , y
    )
    )
    # =
    > a
    r
    r
    a
    y
    (
    [
    5
    , 1
    0
    , 1
    0
    0
    ]
    )

    View full-size slide

  101. # N
    u
    m
    b
    e
    r
    !
    j
    d = t
    o
    d
    a
    y
    (
    )
    p = e
    a
    r
    t
    h
    (
    j
    d
    )
    .
    o
    b
    s
    e
    r
    v
    e
    (
    p
    l
    a
    n
    e
    t
    )
    # V
    e
    c
    t
    o
    r
    !
    j
    d = d
    a
    t
    e
    _
    r
    a
    n
    g
    e
    (
    '
    1
    9
    8
    0
    /
    1
    /
    1
    '
    , '
    2
    0
    1
    0
    /
    1
    /
    1
    '
    , 1
    .
    0
    )
    p = e
    a
    r
    t
    h
    (
    j
    d
    )
    .
    o
    b
    s
    e
    r
    v
    e
    (
    p
    l
    a
    n
    e
    t
    )

    View full-size slide

  102. But scalar style is hard to
    maintain across a large code base!
    n = c
    o
    m
    p
    u
    t
    e
    _
    n
    u
    t
    a
    t
    i
    o
    n
    (
    j
    d
    )
    p = c
    o
    m
    p
    u
    t
    e
    _
    p
    r
    e
    c
    e
    s
    s
    i
    o
    n
    (
    j
    d
    .
    t
    d
    b
    )
    f = J
    2
    0
    0
    0
    _
    t
    o
    _
    I
    C
    R
    S
    t = e
    i
    n
    s
    u
    m
    (
    '
    j
    i
    n
    ,
    k
    j
    n
    -
    >
    i
    k
    n
    '
    , n
    , p
    )
    t = e
    i
    n
    s
    u
    m
    (
    '
    i
    j
    n
    ,
    j
    k
    -
    >
    i
    k
    n
    '
    , t
    , f
    )
    p
    o
    s = e
    i
    n
    s
    u
    m
    (
    '
    i
    n
    ,
    i
    j
    n
    -
    >
    j
    n
    '
    , p
    o
    s
    , t
    )
    v
    e
    l = e
    i
    n
    s
    u
    m
    (
    '
    i
    n
    ,
    i
    j
    n
    -
    >
    j
    n
    '
    , v
    e
    l
    , t
    )

    View full-size slide

  103. One last hint
    Vi Hart says τ = 2ᵰ

    View full-size slide

  104. Q:
    Should dunder-init import things?
    # _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y
    f
    r
    o
    m .
    e
    a
    r
    t
    h
    l
    i
    b i
    m
    p
    o
    r
    t T
    o
    p
    o
    s
    f
    r
    o
    m .
    p
    l
    a
    n
    e
    t
    s i
    m
    p
    o
    r
    t J
    u
    p
    i
    t
    e
    r
    to allow
    f
    r
    o
    m s
    k
    y
    f
    i
    e
    l
    d i
    m
    p
    o
    r
    t T
    o
    p
    o
    s
    , J
    u
    p
    i
    t
    e
    r

    View full-size slide

  105. Pro
    Simple way to surface
    your primary interface

    View full-size slide

  106. Cons
    Innocent i
    m
    p
    o
    r
    t s
    k
    y
    f
    i
    e
    l
    d
    winds up importing everything

    View full-size slide

  107. Cons
    Imports do not match messages
    f
    r
    o
    m s
    k
    y
    f
    i
    e
    l
    d i
    m
    p
    o
    r
    t T
    o
    p
    o
    s
    p
    r
    i
    n
    t t
    y
    p
    e
    (
    T
    o
    p
    o
    s
    )
    # =
    > <
    c
    l
    a
    s
    s '
    s
    k
    y
    f
    i
    e
    l
    d
    .
    c
    o
    o
    r
    d
    i
    n
    a
    t
    e
    s
    .
    T
    o
    p
    o
    s
    '
    >

    View full-size slide

  108. Skyfield: decided against
    Not a one-trick-pony library
    Teach package structure
    Can always s
    k
    y
    f
    i
    e
    l
    d
    .
    a
    p
    i

    View full-size slide

  109. Opinionated defaults

    View full-size slide

  110. Should Skyfield include
    an ephemeris by default?
    s
    e
    t
    u
    p
    (

    i
    n
    s
    t
    a
    l
    l
    _
    r
    e
    q
    u
    i
    r
    e
    s
    =
    [
    '
    d
    e
    4
    2
    1
    '
    ]
    ,
    )

    View full-size slide

  111. Avoid Contrived tests

    View full-size slide

  112. How do you test both
    branches of the i
    f
    ?
    d
    e
    f f
    (
    .
    .
    .
    )
    :

    # 2
    0 l
    i
    n
    e
    s o
    f c
    o
    d
    e

    i
    f f
    i
    n
    a
    l
    _
    d
    e
    c
    i
    s
    i
    o
    n
    (
    )
    :
    a
    c
    t
    i
    o
    n
    1
    e
    l
    s
    e
    :
    a
    c
    t
    i
    o
    n
    2

    View full-size slide

  113. Do not contrive complex calls
    to f
    (
    ) that exercise both branches
    Instead factor the function
    into smaller pieces that can be
    exercised separately

    View full-size slide

  114. Final trick
    Choosing a support forum

    View full-size slide

  115. Mailing list? Web forum?
    What happens 4 years later when
    someone has the same question?

    View full-size slide

  116. Answer can be difficult to find
    within a long discussion thread

    View full-size slide

  117. Answers go out of date

    View full-size slide

  118. Stack Overflow!
    Answer lives on same page
    Out-of-date answers get fixed

    View full-size slide

  119. The last mile

    View full-size slide

  120. The last mile
    Lesson #1

    View full-size slide

  121. New GitHub issue

    View full-size slide

  122. “PyEphem works from ssh but it
    does not work from the web”

    View full-size slide

  123. I was unhappy
    Hurts project reputation
    Issue says “you are wrong”

    View full-size slide

  124. “Can you help me?”

    View full-size slide

  125. “This question is not a good
    fit to our Q&A format”

    View full-size slide

  126. “Can you help me?”

    View full-size slide

  127. i
    m
    p
    o
    r
    t s
    y
    s
    o
    p
    e
    n
    (
    '
    /
    t
    m
    p
    /
    e
    m
    e
    r
    g
    e
    n
    c
    y
    .
    l
    o
    g
    '
    , '
    w
    '
    )
    .
    w
    r
    i
    t
    e
    (
    s
    t
    r
    (
    s
    y
    s
    .
    p
    a
    t
    h
    ) + '
    \
    n
    '
    )

    View full-size slide

  128. i
    m
    p
    o
    r
    t s
    y
    s
    s
    y
    s
    .
    p
    a
    t
    h
    .
    a
    p
    p
    e
    n
    d
    (
    '
    /
    h
    o
    m
    e
    /
    a
    s
    t
    r
    o
    n
    o
    m
    i
    a
    /
    .
    l
    o
    c
    a
    l
    /
    l
    i
    b
    6
    4
    '
    '
    /
    p
    y
    t
    h
    o
    n
    2
    .
    6
    /
    s
    i
    t
    e
    -
    p
    a
    c
    k
    a
    g
    e
    s
    '
    )
    It worked

    View full-size slide

  129. “FYI the support solution is to keep
    that emergency code in every script”

    View full-size slide

  130. “Brandon, thanks a lot for your help.
    PyEphem is great.”

    View full-size slide

  131. The last mile
    Lesson #2

    View full-size slide

  132. October 2012 — I put NOVAS on PyPI
    p
    i
    p i
    n
    s
    t
    a
    l
    l n
    o
    v
    a
    s

    View full-size slide

  133. An email
    “I have been fiddling around
    with NOVAS_Py-3.1 and have
    had some problems…”
    (a list of bugs followed)

    View full-size slide

  134. The temptation
    Not my library — not my problem
    Provide the email address of USNO?
    Forward it to them myself?

    View full-size slide

  135. A
    s
    s
    i
    s
    t
    a
    n
    t D
    i
    r
    e
    c
    t
    o
    r f
    o
    r E
    x
    p
    l
    o
    r
    a
    t
    i
    o
    n
    .
    .
    .
    @
    n
    a
    s
    a
    .
    g
    o
    v

    View full-size slide

  136. USNO → me → NASA

    View full-size slide

  137. The last mile

    View full-size slide

  138. The last mile
    In open source, the
    final component of the API
    is very often you!
    You are the the API of last resort

    View full-size slide

  139. The last mile
    In open source, the
    final component of the API
    is very often you!
    You are the the API of last resort
    @brandon_rhodes
    Thank you very much!

    View full-size slide