Upgrade to Pro — share decks privately, control downloads, hide ads and more …

API Design Tips

daniellindsley
September 04, 2012

API Design Tips

Given at DjangoCon 2012. A collection of ideas (built through the pain of experience) on how to improve the code you write.

daniellindsley

September 04, 2012
Tweet

More Decks by daniellindsley

Other Decks in Technology

Transcript

  1. API Design Tips

    View full-size slide

  2. Who?
    •Daniel Lindsley

    View full-size slide

  3. Who?
    •Daniel Lindsley
    •Toast Driven ( ) is my business

    View full-size slide

  4. Who?
    •Daniel Lindsley
    •Toast Driven ( ) is my business
    •Wrote a couple Django apps
    •Haystack
    •Tastypie

    View full-size slide

  5. What?
    •Not HTTP APIs

    View full-size slide

  6. What?
    •Not HTTP APIs
    •(though I love those. seriously. come
    talk to me about them...)

    View full-size slide

  7. What?
    •Programmatic APIs

    View full-size slide

  8. What?
    •Programmatic APIs
    •Think libraries
    •Especially ones you hand off to
    other people...

    View full-size slide

  9. Why?
    •Think about how many times you’ve
    started with someone else’s library...

    View full-size slide

  10. Why?
    •Think about how many times you’ve
    started with someone else’s library...
    •Used it some...

    View full-size slide

  11. Why?
    •Think about how many times you’ve
    started with someone else’s library...
    •Used it some...
    •Then wanted to strangle them for the
    hoops they made you jump through

    View full-size slide

  12. Why?
    •Other people use your code all the
    time.

    View full-size slide

  13. Why?
    •Other people use your code all the
    time.
    •They might be wanting to strangle you!

    View full-size slide

  14. Why?
    •Other people use your code all the
    time.
    •They might be wanting to strangle you!
    •It might even be future you wanting to
    strangle past you.

    View full-size slide

  15. Why?
    •Nice APIs beget Happiness

    View full-size slide

  16. Why?
    •Nice APIs beget Happiness
    •Happiness begets Recommendations

    View full-size slide

  17. Why?
    •Nice APIs beget Happiness
    •Happiness begets Recommendations
    •Recommendations beget Users/
    Community

    View full-size slide

  18. You can’t make everyone
    happy by default.
    You should still have sane defaults.

    View full-size slide

  19. But more people will be
    happy if they can tweak it.
    You can bet they’ll need to.

    View full-size slide

  20. And most people will be
    happy if it’s easy to tweak.
    I don’t have a clever followup here.

    View full-size slide

  21. No copy-paste should
    be needed.
    Boilerplate sucks.

    View full-size slide

  22. They shouldn’t have to
    constantly refer to the
    docs.
    Especially not for things they use
    all the freaking time.

    View full-size slide

  23. Good docs matter.
    Saves you (support) &
    them (implementation) time.
    Everyone wins.

    View full-size slide

  24. Real World™ use is the
    best sanity check.
    Cleverness goes here.

    View full-size slide

  25. Someone is going to do
    something weird/insane
    with your code.
    It’s inevitable, so design for it up front.

    View full-size slide

  26. Approaches on
    Design

    View full-size slide

  27. Approaches On
    Design
    •Common Methodologies:

    View full-size slide

  28. Approaches On
    Design
    •Common Methodologies:
    •Bottom-up

    View full-size slide

  29. Approaches On
    Design
    •Common Methodologies:
    •Bottom-up
    •Top-down

    View full-size slide

  30. Approaches On
    Design
    •Common Methodologies:
    •Bottom-up
    •Top-down
    •WARNING: OPINIONS FOLLOW

    View full-size slide

  31. Approaches On
    Design
    •Bottom-up sucks.

    View full-size slide

  32. Approaches On
    Design
    •Bottom-up sucks.
    •Sure, you built little pieces that work.

    View full-size slide

  33. Approaches On
    Design
    •Bottom-up sucks.
    •Sure, you built little pieces that work.
    •But do they work well together?

    View full-size slide

  34. Approaches On
    Design
    •Bottom-up sucks.
    •Sure, you built little pieces that work.
    •But do they work well together?
    •Likely not.
    •HOW IS BABBY PHP FORMED

    View full-size slide

  35. Approaches On
    Design
    •Top-down feels better.

    View full-size slide

  36. Approaches On
    Design
    •Top-down feels better.
    •Everything fits together right.

    View full-size slide

  37. Approaches On
    Design
    •Top-down feels better.
    •Everything fits together right.
    •Less duplication.

    View full-size slide

  38. Approaches On
    Design
    •Top-down feels better.
    •Everything fits together right.
    •Less duplication.
    •Resist the urge to duct tape.

    View full-size slide

  39. Bonus to Top-Down?

    View full-size slide

  40. Approaches On
    Design
    •With (some) instant TDD, you get your
    tests started from the get-go.

    View full-size slide

  41. Approaches On
    Design
    •With (some) instant TDD, you get your
    tests started from the get-go.
    •Fewer massive, painful refactorings
    down the road.

    View full-size slide

  42. Things You Should Do

    View full-size slide

  43. Small Components
    Worked for UNIX, it’ll work for you.

    View full-size slide

  44. Small Components
    class DoEverything(object):
    # ...
    def update_cache(self, timeout=600):
    # Stuff...
    def post_to_twitter(self, user):
    # More stuff...
    def make_me_a_sandwich(self, sudo=False):
    # Even more stuff...

    View full-size slide

  45. Small Components
    class Cachable(object):
    timeout = 600
    def update(self):
    # Cache myself...
    class SocialPosts(object):
    def twitter(self, user):
    # Post a tweet
    class SandwichMaker(object):
    def __init__(self, sudo=False):
    self.sudo = sudo
    def make(self):
    # Maybe yes, maybe no.

    View full-size slide

  46. Composition
    >=
    Inheritance
    Why do the work yourself when you can
    delegate?

    View full-size slide

  47. Composition >=
    Inheritance
    class CachedView(View):
    def get(self, request, **kwargs):
    # Cache the output & serve.
    class GhettoAPIView(View):
    def get(self, request, **kwargs):
    # Return either JSON or HTML.
    # Now try to use both. :/
    class MyView(CachedView, GhettoAPIView):
    # Oh crap, they stomp on each other.
    pass

    View full-size slide

  48. Composition >=
    Inheritance
    class MyView(BetterView):
    def get(self, **kwargs):
    response = self.preprocess()
    # Do stuff.
    return self.render('template.html', {})
    def preprocess(self):
    # Delegate!
    cached = ViewCache()
    return cached.get(self, self.request)
    def render(self, template, context):
    output = GhettoAPI(self.request)
    return output.render(template, context)

    View full-size slide

  49. Reflection
    If data can flow one way, add the
    opposite direction as well.

    View full-size slide

  50. Reflection
    class FakeDate(BadDate):
    def to_iso8801(self):
    return '-'.join(
    self.year,
    self.month,
    self.day
    )

    View full-size slide

  51. Reflection
    class FakeDate(BadDate):
    def to_iso8801(self):
    return '-'.join(
    self.year,
    self.month,
    self.day
    )
    def from_iso8801(self, date_string):
    date_bits = date_string.split('-')
    self.year = date_bits[0]
    self.month = date_bits[1]
    self.day = date_bits[2]

    View full-size slide

  52. Broad Familiarity
    If it’s a similar task to something else they’ll know,
    mimic that something else.

    View full-size slide

  53. Broad Familiarity
    def go_search(engine_url, query, limit=10
    offset=0, just_values=False):
    # Hit the search engine...
    return results
    def crazy_raw_query(engine_url, raw_query):
    # Hope & pray the query is OK.
    return raw_results
    # Usage:
    kwargs = {
    'limit': 20,
    'offset': 20,
    }
    results = go_search('http://...', 'banana')

    View full-size slide

  54. Broad Familiarity
    class SearchQuerySet(object):
    # Make it look like ``QuerySet``.
    def __getitem__(self, key):
    # Handle the slice here.
    def filter(self, **kwargs):
    # Apply filters.
    def raw(self, query):
    # Do a raw query.
    # Usage:
    sqs = SearchQuerySet().filter(text='banana')
    results = sqs[20:20]

    View full-size slide

  55. Narrow Familiarity
    Call signatures matter.

    View full-size slide

  56. Narrow Familiarity
    def go_search(engine_url, query, limit=10
    offset=0, just_values=False):
    # Do stuff.
    return results
    def execute_raw_query(raw_query, offset,
    engine_url):
    # Do a raw query here.
    return {
    'search_results': results,
    }

    View full-size slide

  57. Narrow Familiarity
    def query(engine_url, query, limit=10
    offset=0, just_values=False):
    # Do stuff.
    return {
    'results': results,
    'total': total_count,
    }
    def raw_query(engine_url, query, limit=10
    offset=0, just_values=False):
    # Do a raw query here.
    return {
    'results': results,
    'total': total_count,
    }

    View full-size slide

  58. Protocol
    Gently “suggest” things behave the same way.
    With a stick.

    View full-size slide

  59. Protocol
    class CoolKidBackend(object):
    # Whatever, as if we care.
    pass
    class RedisBackend(CoolKidBackend):
    def get(self, key):
    return self.conn.hget(key)
    class RiakBackend(CoolKidBackend):
    def get_hash(self, bucket, key):
    return self._conn.get(bucket, key)
    # You can't transparently switch backends.
    # Welp, time to go rewrite everything.

    View full-size slide

  60. Protocol
    class CoolKidBackend(object):
    def __init__(self, conn_string):
    # Parse the bucket out of
    # the conn_string, if present.
    def get(self, key):
    raise NotImplementedError('...')
    class RedisBackend(CoolKidBackend):
    def get(self, key):
    return self.conn.hget(key)
    class RiakBackend(CoolKidBackend):
    def get(self, key):
    return self._conn.get(...)

    View full-size slide

  61. Assume The Worst™!
    Don’t code for just the easy case.

    View full-size slide

  62. Assume The Worst™!
    def parse_name(name):
    # It'll be fine. Names are always
    # " "
    # and ASCII, right?
    return name.split(' ')

    View full-size slide

  63. Assume The Worst™!
    def parse_name(name, separator=' ',
    middlename_present=False,
    allowed_suffixes=SUFFIXES):
    if ',' in name:
    maybe_last_name = name.split(',', 1)
    if maybe_last_name in allowed_suffixes:
    # ...
    return {
    'original': name,
    'confidence': confidence_level * 100,
    'first_name': first,
    # ...
    }

    View full-size slide

  64. Things You Should
    NOT Do

    View full-size slide

  65. Stop At A Low Level
    “This Low Level API ought to be good enough.”
    Yeah, right. And 640Kb was
    totally enough for everyone.

    View full-size slide

  66. Stop At A Low Level
    def get_page(socket, timeout):
    # Use the open socket to fetch
    # the web page. It's a open
    # socket to the correct place,
    # right?

    View full-size slide

  67. Stop At A Low Level
    def get_raw_page(socket, timeout=60):
    # Do the actual fetch.
    return headers, content
    def get_page(url, timeout=None,
    fetcher=get_raw_page):
    # Validate the URL.
    # Open a socket.
    head, content = fetcher(my_socket)
    if head.status == 500:
    raise OhNoes('...')
    # ...

    View full-size slide

  68. Wildly Different
    Return Values
    “Hm, does this return an integer or a dictionary?”

    View full-size slide

  69. Wildly Different
    Return Values
    def query(engine_url, query, limit=10):
    # Make the user deal with the raw data.
    results = [res for res in results if res]
    return results, total
    def raw_query(engine_url, query, limit=10):
    # Make the user deal with the raw data.
    return {
    'search_results': results,
    'count': total,
    }

    View full-size slide

  70. Wildly Different
    Return Values
    def query(engine_url, query, limit=10):
    # We're doing a search query.
    # Return sane, similar structures.
    return {
    'results': results,
    'total': total_count,
    }
    def raw_query(engine_url, query, limit=10):
    # We're doing a search query.
    # Return sane, similar structures.
    return {
    'results': results,
    'total': total_count,
    }

    View full-size slide

  71. Useless
    “Implementation”
    Code
    If it can’t do anything for itself,
    it’s code without purpose.

    View full-size slide

  72. Useless
    “Implementation” Code
    class BaseBackend(object):
    # Full implementation.
    def __init__(self, url):
    raise NotImplementedError('...')
    def save(self, filename, data):
    raise NotImplementedError('...')
    def load(self, filename):
    raise NotImplementedError('...')

    View full-size slide

  73. Useless
    “Implementation” Code
    class FileBackend(object):
    # Does useful things. Subclasses can
    # override less.
    def __init__(self, url):
    self.url = url
    def save(self, filename, data):
    the_file = open(filename, 'w')
    return json.dump(the_file, data)
    def load(self, filename):
    data = open(filename)
    return json.load(data)

    View full-size slide

  74. If It’s Difficult To Test...
    ...you screwed up.
    But maybe you can fix it.

    View full-size slide

  75. If It’s Difficult To Test...
    def current_temperature(station):
    # Oh yeah, this totally works BTW.
    url = 'http://w1.weather.gov/obhistory/' + \
    '%s.html' % station
    resp = requests.get(url)
    data = pq(resp.content)
    return float(data('table').eq(3)\
    .find('tr').eq(4)\
    .find('td').eq(6).text())

    View full-size slide

  76. If It’s Difficult To Test...
    class Weather(object):
    def build_url(self, station):
    return 'http://w1.weather.gov/obhistory/' + \
    '%s.html' % station
    def fetch_page(self, url):
    return requests.get(url).content
    def parse_page(self, content):
    return pq(resp.content)
    def last_conditions(self, data):
    return data('table').eq(3).find('tr').eq(4)
    def current_temperature(self, station):
    url = self.build_url(station)
    content = self.fetch_page(url)
    data = self.parse_page(content)
    return float(self.last_conditions(data)\
    .find('td').eq(6).text())

    View full-size slide

  77. django-Specific
    Topics

    View full-size slide

  78. Pluggable Backend
    ALL THE THINGS

    View full-size slide

  79. Internationalize
    ALL THE THINGS

    View full-size slide

  80. Dynamically Loadable
    Classes/Code
    60% more error-handling, every time.

    View full-size slide

  81. Declarative Syntax
    Metaclasses: Call your doctor if headaches
    last more than 4 hours.

    View full-size slide

  82. Don’t metaclass
    ALL THE THINGS

    View full-size slide

  83. The ORM
    Love it or hate it, there’s some great
    learning opportunities there to be had.

    View full-size slide

  84. Very Little Global State
    # Only defaults or things that absolute
    # have to be module-level.
    BASE_URL = 'http://w1.weather.gov/obhistory/'
    TIMEOUT = 10
    # Make them defaults but allow passing
    # different in their place.
    def fetch_page(self, url=BASE_URL,
    timeout=TIMEOUT):
    return requests.get(
    url,
    timeout=timeout
    ).content

    View full-size slide

  85. Decrease Reliance
    On ``self``
    class Weather(object):
    def build_url(self, station):
    # Make a URL.
    def fetch_page(self, url):
    # Get a page
    def parse_page(self, content):
    # Use the page content.
    def current_temperature(self, station):
    url = self.build_url(station)
    content = self.fetch_page(url)
    data = self.parse_page(content)
    # You could be using ``self.`` on
    # these, but by allowing data to be passed
    # in, you make it easier to test.

    View full-size slide

  86. Resist The Urge
    To Use Magic
    Be explicit first, then add shortcuts.
    (which can be a little more magical)

    View full-size slide

  87. So...
    •Use the Golden Rule.

    View full-size slide

  88. So...
    •Use the Golden Rule.
    •Consistency is key.

    View full-size slide

  89. So...
    •Use the Golden Rule.
    •Consistency is key.
    •Plan for the worst & include sweet
    shortcuts for the best.

    View full-size slide

  90. So...
    •Use the Golden Rule.
    •Consistency is key.
    •Plan for the worst & include sweet
    shortcuts for the best.
    •Write a thing you’d to use...

    View full-size slide

  91. So...
    •Use the Golden Rule.
    •Consistency is key.
    •Plan for the worst & include sweet
    shortcuts for the best.
    •Write a thing you’d to use...
    •...then make it even better.

    View full-size slide

  92. I’m Daniel Lindsley
    of Toast Driven
    @toastdriven
    http://toastdriven.com/

    View full-size slide