Upgrade to Pro — share decks privately, control downloads, hide ads and more …

API Design Tips

daniellindsley
September 04, 2012

API Design Tips

Given at DjangoCon 2012. A collection of ideas (built through the pain of experience) on how to improve the code you write.

daniellindsley

September 04, 2012
Tweet

More Decks by daniellindsley

Other Decks in Technology

Transcript

  1. API Design Tips

    View Slide

  2. Who?

    View Slide

  3. Who?
    •Daniel Lindsley

    View Slide

  4. Who?
    •Daniel Lindsley
    •Toast Driven ( ) is my business

    View Slide

  5. Who?
    •Daniel Lindsley
    •Toast Driven ( ) is my business
    •Wrote a couple Django apps
    •Haystack
    •Tastypie

    View Slide

  6. What?

    View Slide

  7. What?
    •Not HTTP APIs

    View Slide

  8. What?
    •Not HTTP APIs
    •(though I love those. seriously. come
    talk to me about them...)

    View Slide

  9. What?
    •Programmatic APIs

    View Slide

  10. What?
    •Programmatic APIs
    •Think libraries
    •Especially ones you hand off to
    other people...

    View Slide

  11. Why?

    View Slide

  12. Why?
    •Think about how many times you’ve
    started with someone else’s library...

    View Slide

  13. Why?
    •Think about how many times you’ve
    started with someone else’s library...
    •Used it some...

    View Slide

  14. Why?
    •Think about how many times you’ve
    started with someone else’s library...
    •Used it some...
    •Then wanted to strangle them for the
    hoops they made you jump through

    View Slide

  15. Why?
    •Other people use your code all the
    time.

    View Slide

  16. Why?
    •Other people use your code all the
    time.
    •They might be wanting to strangle you!

    View Slide

  17. Why?
    •Other people use your code all the
    time.
    •They might be wanting to strangle you!
    •It might even be future you wanting to
    strangle past you.

    View Slide

  18. Why?
    •Nice APIs beget Happiness

    View Slide

  19. Why?
    •Nice APIs beget Happiness
    •Happiness begets Recommendations

    View Slide

  20. Why?
    •Nice APIs beget Happiness
    •Happiness begets Recommendations
    •Recommendations beget Users/
    Community

    View Slide

  21. Philosophy

    View Slide

  22. You can’t make everyone
    happy by default.
    You should still have sane defaults.

    View Slide

  23. But more people will be
    happy if they can tweak it.
    You can bet they’ll need to.

    View Slide

  24. And most people will be
    happy if it’s easy to tweak.
    I don’t have a clever followup here.

    View Slide

  25. No copy-paste should
    be needed.
    Boilerplate sucks.

    View Slide

  26. They shouldn’t have to
    constantly refer to the
    docs.
    Especially not for things they use
    all the freaking time.

    View Slide

  27. Good docs matter.
    Saves you (support) &
    them (implementation) time.
    Everyone wins.

    View Slide

  28. Real World™ use is the
    best sanity check.
    Cleverness goes here.

    View Slide

  29. Someone is going to do
    something weird/insane
    with your code.
    It’s inevitable, so design for it up front.

    View Slide

  30. Approaches on
    Design

    View Slide

  31. Approaches On
    Design
    •Common Methodologies:

    View Slide

  32. Approaches On
    Design
    •Common Methodologies:
    •Bottom-up

    View Slide

  33. Approaches On
    Design
    •Common Methodologies:
    •Bottom-up
    •Top-down

    View Slide

  34. Approaches On
    Design
    •Common Methodologies:
    •Bottom-up
    •Top-down
    •WARNING: OPINIONS FOLLOW

    View Slide

  35. Approaches On
    Design
    •Bottom-up sucks.

    View Slide

  36. Approaches On
    Design
    •Bottom-up sucks.
    •Sure, you built little pieces that work.

    View Slide

  37. Approaches On
    Design
    •Bottom-up sucks.
    •Sure, you built little pieces that work.
    •But do they work well together?

    View Slide

  38. Approaches On
    Design
    •Bottom-up sucks.
    •Sure, you built little pieces that work.
    •But do they work well together?
    •Likely not.
    •HOW IS BABBY PHP FORMED

    View Slide

  39. Approaches On
    Design
    •Top-down feels better.

    View Slide

  40. Approaches On
    Design
    •Top-down feels better.
    •Everything fits together right.

    View Slide

  41. Approaches On
    Design
    •Top-down feels better.
    •Everything fits together right.
    •Less duplication.

    View Slide

  42. Approaches On
    Design
    •Top-down feels better.
    •Everything fits together right.
    •Less duplication.
    •Resist the urge to duct tape.

    View Slide

  43. Bonus to Top-Down?

    View Slide

  44. Instant TDD

    View Slide

  45. Approaches On
    Design
    •With (some) instant TDD, you get your
    tests started from the get-go.

    View Slide

  46. Approaches On
    Design
    •With (some) instant TDD, you get your
    tests started from the get-go.
    •Fewer massive, painful refactorings
    down the road.

    View Slide

  47. Things You Should Do

    View Slide

  48. Small Components
    Worked for UNIX, it’ll work for you.

    View Slide

  49. Small Components
    class DoEverything(object):
    # ...
    def update_cache(self, timeout=600):
    # Stuff...
    def post_to_twitter(self, user):
    # More stuff...
    def make_me_a_sandwich(self, sudo=False):
    # Even more stuff...

    View Slide

  50. Small Components
    class Cachable(object):
    timeout = 600
    def update(self):
    # Cache myself...
    class SocialPosts(object):
    def twitter(self, user):
    # Post a tweet
    class SandwichMaker(object):
    def __init__(self, sudo=False):
    self.sudo = sudo
    def make(self):
    # Maybe yes, maybe no.

    View Slide

  51. Composition
    >=
    Inheritance
    Why do the work yourself when you can
    delegate?

    View Slide

  52. Composition >=
    Inheritance
    class CachedView(View):
    def get(self, request, **kwargs):
    # Cache the output & serve.
    class GhettoAPIView(View):
    def get(self, request, **kwargs):
    # Return either JSON or HTML.
    # Now try to use both. :/
    class MyView(CachedView, GhettoAPIView):
    # Oh crap, they stomp on each other.
    pass

    View Slide

  53. Composition >=
    Inheritance
    class MyView(BetterView):
    def get(self, **kwargs):
    response = self.preprocess()
    # Do stuff.
    return self.render('template.html', {})
    def preprocess(self):
    # Delegate!
    cached = ViewCache()
    return cached.get(self, self.request)
    def render(self, template, context):
    output = GhettoAPI(self.request)
    return output.render(template, context)

    View Slide

  54. Reflection
    If data can flow one way, add the
    opposite direction as well.

    View Slide

  55. Reflection
    class FakeDate(BadDate):
    def to_iso8801(self):
    return '-'.join(
    self.year,
    self.month,
    self.day
    )

    View Slide

  56. Reflection
    class FakeDate(BadDate):
    def to_iso8801(self):
    return '-'.join(
    self.year,
    self.month,
    self.day
    )
    def from_iso8801(self, date_string):
    date_bits = date_string.split('-')
    self.year = date_bits[0]
    self.month = date_bits[1]
    self.day = date_bits[2]

    View Slide

  57. Broad Familiarity
    If it’s a similar task to something else they’ll know,
    mimic that something else.

    View Slide

  58. Broad Familiarity
    def go_search(engine_url, query, limit=10
    offset=0, just_values=False):
    # Hit the search engine...
    return results
    def crazy_raw_query(engine_url, raw_query):
    # Hope & pray the query is OK.
    return raw_results
    # Usage:
    kwargs = {
    'limit': 20,
    'offset': 20,
    }
    results = go_search('http://...', 'banana')

    View Slide

  59. Broad Familiarity
    class SearchQuerySet(object):
    # Make it look like ``QuerySet``.
    def __getitem__(self, key):
    # Handle the slice here.
    def filter(self, **kwargs):
    # Apply filters.
    def raw(self, query):
    # Do a raw query.
    # Usage:
    sqs = SearchQuerySet().filter(text='banana')
    results = sqs[20:20]

    View Slide

  60. Narrow Familiarity
    Call signatures matter.

    View Slide

  61. Narrow Familiarity
    def go_search(engine_url, query, limit=10
    offset=0, just_values=False):
    # Do stuff.
    return results
    def execute_raw_query(raw_query, offset,
    engine_url):
    # Do a raw query here.
    return {
    'search_results': results,
    }

    View Slide

  62. Narrow Familiarity
    def query(engine_url, query, limit=10
    offset=0, just_values=False):
    # Do stuff.
    return {
    'results': results,
    'total': total_count,
    }
    def raw_query(engine_url, query, limit=10
    offset=0, just_values=False):
    # Do a raw query here.
    return {
    'results': results,
    'total': total_count,
    }

    View Slide

  63. Protocol
    Gently “suggest” things behave the same way.
    With a stick.

    View Slide

  64. Protocol
    class CoolKidBackend(object):
    # Whatever, as if we care.
    pass
    class RedisBackend(CoolKidBackend):
    def get(self, key):
    return self.conn.hget(key)
    class RiakBackend(CoolKidBackend):
    def get_hash(self, bucket, key):
    return self._conn.get(bucket, key)
    # You can't transparently switch backends.
    # Welp, time to go rewrite everything.

    View Slide

  65. Protocol
    class CoolKidBackend(object):
    def __init__(self, conn_string):
    # Parse the bucket out of
    # the conn_string, if present.
    def get(self, key):
    raise NotImplementedError('...')
    class RedisBackend(CoolKidBackend):
    def get(self, key):
    return self.conn.hget(key)
    class RiakBackend(CoolKidBackend):
    def get(self, key):
    return self._conn.get(...)

    View Slide

  66. Assume The Worst™!
    Don’t code for just the easy case.

    View Slide

  67. Assume The Worst™!
    def parse_name(name):
    # It'll be fine. Names are always
    # " "
    # and ASCII, right?
    return name.split(' ')

    View Slide

  68. Assume The Worst™!
    def parse_name(name, separator=' ',
    middlename_present=False,
    allowed_suffixes=SUFFIXES):
    if ',' in name:
    maybe_last_name = name.split(',', 1)
    if maybe_last_name in allowed_suffixes:
    # ...
    return {
    'original': name,
    'confidence': confidence_level * 100,
    'first_name': first,
    # ...
    }

    View Slide

  69. Things You Should
    NOT Do

    View Slide

  70. Stop At A Low Level
    “This Low Level API ought to be good enough.”
    Yeah, right. And 640Kb was
    totally enough for everyone.

    View Slide

  71. Stop At A Low Level
    def get_page(socket, timeout):
    # Use the open socket to fetch
    # the web page. It's a open
    # socket to the correct place,
    # right?

    View Slide

  72. Stop At A Low Level
    def get_raw_page(socket, timeout=60):
    # Do the actual fetch.
    return headers, content
    def get_page(url, timeout=None,
    fetcher=get_raw_page):
    # Validate the URL.
    # Open a socket.
    head, content = fetcher(my_socket)
    if head.status == 500:
    raise OhNoes('...')
    # ...

    View Slide

  73. Wildly Different
    Return Values
    “Hm, does this return an integer or a dictionary?”

    View Slide

  74. Wildly Different
    Return Values
    def query(engine_url, query, limit=10):
    # Make the user deal with the raw data.
    results = [res for res in results if res]
    return results, total
    def raw_query(engine_url, query, limit=10):
    # Make the user deal with the raw data.
    return {
    'search_results': results,
    'count': total,
    }

    View Slide

  75. Wildly Different
    Return Values
    def query(engine_url, query, limit=10):
    # We're doing a search query.
    # Return sane, similar structures.
    return {
    'results': results,
    'total': total_count,
    }
    def raw_query(engine_url, query, limit=10):
    # We're doing a search query.
    # Return sane, similar structures.
    return {
    'results': results,
    'total': total_count,
    }

    View Slide

  76. Useless
    “Implementation”
    Code
    If it can’t do anything for itself,
    it’s code without purpose.

    View Slide

  77. Useless
    “Implementation” Code
    class BaseBackend(object):
    # Full implementation.
    def __init__(self, url):
    raise NotImplementedError('...')
    def save(self, filename, data):
    raise NotImplementedError('...')
    def load(self, filename):
    raise NotImplementedError('...')

    View Slide

  78. Useless
    “Implementation” Code
    class FileBackend(object):
    # Does useful things. Subclasses can
    # override less.
    def __init__(self, url):
    self.url = url
    def save(self, filename, data):
    the_file = open(filename, 'w')
    return json.dump(the_file, data)
    def load(self, filename):
    data = open(filename)
    return json.load(data)

    View Slide

  79. If It’s Difficult To Test...
    ...you screwed up.
    But maybe you can fix it.

    View Slide

  80. If It’s Difficult To Test...
    def current_temperature(station):
    # Oh yeah, this totally works BTW.
    url = 'http://w1.weather.gov/obhistory/' + \
    '%s.html' % station
    resp = requests.get(url)
    data = pq(resp.content)
    return float(data('table').eq(3)\
    .find('tr').eq(4)\
    .find('td').eq(6).text())

    View Slide

  81. If It’s Difficult To Test...
    class Weather(object):
    def build_url(self, station):
    return 'http://w1.weather.gov/obhistory/' + \
    '%s.html' % station
    def fetch_page(self, url):
    return requests.get(url).content
    def parse_page(self, content):
    return pq(resp.content)
    def last_conditions(self, data):
    return data('table').eq(3).find('tr').eq(4)
    def current_temperature(self, station):
    url = self.build_url(station)
    content = self.fetch_page(url)
    data = self.parse_page(content)
    return float(self.last_conditions(data)\
    .find('td').eq(6).text())

    View Slide

  82. django-Specific
    Topics

    View Slide

  83. Pluggable Backend
    ALL THE THINGS

    View Slide

  84. Internationalize
    ALL THE THINGS

    View Slide

  85. Dynamically Loadable
    Classes/Code
    60% more error-handling, every time.

    View Slide

  86. Declarative Syntax
    Metaclasses: Call your doctor if headaches
    last more than 4 hours.

    View Slide

  87. Don’t metaclass
    ALL THE THINGS

    View Slide

  88. The ORM
    Love it or hate it, there’s some great
    learning opportunities there to be had.

    View Slide

  89. Other Ideas

    View Slide

  90. Very Little Global State
    # Only defaults or things that absolute
    # have to be module-level.
    BASE_URL = 'http://w1.weather.gov/obhistory/'
    TIMEOUT = 10
    # Make them defaults but allow passing
    # different in their place.
    def fetch_page(self, url=BASE_URL,
    timeout=TIMEOUT):
    return requests.get(
    url,
    timeout=timeout
    ).content

    View Slide

  91. Decrease Reliance
    On ``self``
    class Weather(object):
    def build_url(self, station):
    # Make a URL.
    def fetch_page(self, url):
    # Get a page
    def parse_page(self, content):
    # Use the page content.
    def current_temperature(self, station):
    url = self.build_url(station)
    content = self.fetch_page(url)
    data = self.parse_page(content)
    # You could be using ``self.`` on
    # these, but by allowing data to be passed
    # in, you make it easier to test.

    View Slide

  92. Resist The Urge
    To Use Magic
    Be explicit first, then add shortcuts.
    (which can be a little more magical)

    View Slide

  93. So...

    View Slide

  94. So...
    •Use the Golden Rule.

    View Slide

  95. So...
    •Use the Golden Rule.
    •Consistency is key.

    View Slide

  96. So...
    •Use the Golden Rule.
    •Consistency is key.
    •Plan for the worst & include sweet
    shortcuts for the best.

    View Slide

  97. So...
    •Use the Golden Rule.
    •Consistency is key.
    •Plan for the worst & include sweet
    shortcuts for the best.
    •Write a thing you’d to use...

    View Slide

  98. So...
    •Use the Golden Rule.
    •Consistency is key.
    •Plan for the worst & include sweet
    shortcuts for the best.
    •Write a thing you’d to use...
    •...then make it even better.

    View Slide

  99. Thanks!

    View Slide

  100. I’m Daniel Lindsley
    of Toast Driven
    @toastdriven
    http://toastdriven.com/

    View Slide