API Design Tips

Ea39e564e226a87b507a00d46e471e10?s=47 daniellindsley
September 04, 2012

API Design Tips

Given at DjangoCon 2012. A collection of ideas (built through the pain of experience) on how to improve the code you write.

Ea39e564e226a87b507a00d46e471e10?s=128

daniellindsley

September 04, 2012
Tweet

Transcript

  1. API Design Tips

  2. Who?

  3. Who? •Daniel Lindsley

  4. Who? •Daniel Lindsley •Toast Driven ( ) is my business

  5. Who? •Daniel Lindsley •Toast Driven ( ) is my business

    •Wrote a couple Django apps •Haystack •Tastypie
  6. What?

  7. What? •Not HTTP APIs

  8. What? •Not HTTP APIs •(though I love those. seriously. come

    talk to me about them...)
  9. What? •Programmatic APIs

  10. What? •Programmatic APIs •Think libraries •Especially ones you hand off

    to other people...
  11. Why?

  12. Why? •Think about how many times you’ve started with someone

    else’s library...
  13. Why? •Think about how many times you’ve started with someone

    else’s library... •Used it some...
  14. Why? •Think about how many times you’ve started with someone

    else’s library... •Used it some... •Then wanted to strangle them for the hoops they made you jump through
  15. Why? •Other people use your code all the time.

  16. Why? •Other people use your code all the time. •They

    might be wanting to strangle you!
  17. Why? •Other people use your code all the time. •They

    might be wanting to strangle you! •It might even be future you wanting to strangle past you.
  18. Why? •Nice APIs beget Happiness

  19. Why? •Nice APIs beget Happiness •Happiness begets Recommendations

  20. Why? •Nice APIs beget Happiness •Happiness begets Recommendations •Recommendations beget

    Users/ Community
  21. Philosophy

  22. You can’t make everyone happy by default. You should still

    have sane defaults.
  23. But more people will be happy if they can tweak

    it. You can bet they’ll need to.
  24. And most people will be happy if it’s easy to

    tweak. I don’t have a clever followup here.
  25. No copy-paste should be needed. Boilerplate sucks.

  26. They shouldn’t have to constantly refer to the docs. Especially

    not for things they use all the freaking time.
  27. Good docs matter. Saves you (support) & them (implementation) time.

    Everyone wins.
  28. Real World™ use is the best sanity check. Cleverness goes

    here.
  29. Someone is going to do something weird/insane with your code.

    It’s inevitable, so design for it up front.
  30. Approaches on Design

  31. Approaches On Design •Common Methodologies:

  32. Approaches On Design •Common Methodologies: •Bottom-up

  33. Approaches On Design •Common Methodologies: •Bottom-up •Top-down

  34. Approaches On Design •Common Methodologies: •Bottom-up •Top-down •WARNING: OPINIONS FOLLOW

  35. Approaches On Design •Bottom-up sucks.

  36. Approaches On Design •Bottom-up sucks. •Sure, you built little pieces

    that work.
  37. Approaches On Design •Bottom-up sucks. •Sure, you built little pieces

    that work. •But do they work well together?
  38. Approaches On Design •Bottom-up sucks. •Sure, you built little pieces

    that work. •But do they work well together? •Likely not. •HOW IS BABBY PHP FORMED
  39. Approaches On Design •Top-down feels better.

  40. Approaches On Design •Top-down feels better. •Everything fits together right.

  41. Approaches On Design •Top-down feels better. •Everything fits together right.

    •Less duplication.
  42. Approaches On Design •Top-down feels better. •Everything fits together right.

    •Less duplication. •Resist the urge to duct tape.
  43. Bonus to Top-Down?

  44. Instant TDD

  45. Approaches On Design •With (some) instant TDD, you get your

    tests started from the get-go.
  46. Approaches On Design •With (some) instant TDD, you get your

    tests started from the get-go. •Fewer massive, painful refactorings down the road.
  47. Things You Should Do

  48. Small Components Worked for UNIX, it’ll work for you.

  49. Small Components class DoEverything(object): # ... def update_cache(self, timeout=600): #

    Stuff... def post_to_twitter(self, user): # More stuff... def make_me_a_sandwich(self, sudo=False): # Even more stuff...
  50. Small Components class Cachable(object): timeout = 600 def update(self): #

    Cache myself... class SocialPosts(object): def twitter(self, user): # Post a tweet class SandwichMaker(object): def __init__(self, sudo=False): self.sudo = sudo def make(self): # Maybe yes, maybe no.
  51. Composition >= Inheritance Why do the work yourself when you

    can delegate?
  52. Composition >= Inheritance class CachedView(View): def get(self, request, **kwargs): #

    Cache the output & serve. class GhettoAPIView(View): def get(self, request, **kwargs): # Return either JSON or HTML. # Now try to use both. :/ class MyView(CachedView, GhettoAPIView): # Oh crap, they stomp on each other. pass
  53. Composition >= Inheritance class MyView(BetterView): def get(self, **kwargs): response =

    self.preprocess() # Do stuff. return self.render('template.html', {}) def preprocess(self): # Delegate! cached = ViewCache() return cached.get(self, self.request) def render(self, template, context): output = GhettoAPI(self.request) return output.render(template, context)
  54. Reflection If data can flow one way, add the opposite

    direction as well.
  55. Reflection class FakeDate(BadDate): def to_iso8801(self): return '-'.join( self.year, self.month, self.day

    )
  56. Reflection class FakeDate(BadDate): def to_iso8801(self): return '-'.join( self.year, self.month, self.day

    ) def from_iso8801(self, date_string): date_bits = date_string.split('-') self.year = date_bits[0] self.month = date_bits[1] self.day = date_bits[2]
  57. Broad Familiarity If it’s a similar task to something else

    they’ll know, mimic that something else.
  58. Broad Familiarity def go_search(engine_url, query, limit=10 offset=0, just_values=False): # Hit

    the search engine... return results def crazy_raw_query(engine_url, raw_query): # Hope & pray the query is OK. return raw_results # Usage: kwargs = { 'limit': 20, 'offset': 20, } results = go_search('http://...', 'banana')
  59. Broad Familiarity class SearchQuerySet(object): # Make it look like ``QuerySet``.

    def __getitem__(self, key): # Handle the slice here. def filter(self, **kwargs): # Apply filters. def raw(self, query): # Do a raw query. # Usage: sqs = SearchQuerySet().filter(text='banana') results = sqs[20:20]
  60. Narrow Familiarity Call signatures matter.

  61. Narrow Familiarity def go_search(engine_url, query, limit=10 offset=0, just_values=False): # Do

    stuff. return results def execute_raw_query(raw_query, offset, engine_url): # Do a raw query here. return { 'search_results': results, }
  62. Narrow Familiarity def query(engine_url, query, limit=10 offset=0, just_values=False): # Do

    stuff. return { 'results': results, 'total': total_count, } def raw_query(engine_url, query, limit=10 offset=0, just_values=False): # Do a raw query here. return { 'results': results, 'total': total_count, }
  63. Protocol Gently “suggest” things behave the same way. With a

    stick.
  64. Protocol class CoolKidBackend(object): # Whatever, as if we care. pass

    class RedisBackend(CoolKidBackend): def get(self, key): return self.conn.hget(key) class RiakBackend(CoolKidBackend): def get_hash(self, bucket, key): return self._conn.get(bucket, key) # You can't transparently switch backends. # Welp, time to go rewrite everything.
  65. Protocol class CoolKidBackend(object): def __init__(self, conn_string): # Parse the bucket

    out of # the conn_string, if present. def get(self, key): raise NotImplementedError('...') class RedisBackend(CoolKidBackend): def get(self, key): return self.conn.hget(key) class RiakBackend(CoolKidBackend): def get(self, key): return self._conn.get(...)
  66. Assume The Worst™! Don’t code for just the easy case.

  67. Assume The Worst™! def parse_name(name): # It'll be fine. Names

    are always # "<first name> <single last name>" # and ASCII, right? return name.split(' ')
  68. Assume The Worst™! def parse_name(name, separator=' ', middlename_present=False, allowed_suffixes=SUFFIXES): if

    ',' in name: maybe_last_name = name.split(',', 1) if maybe_last_name in allowed_suffixes: # ... return { 'original': name, 'confidence': confidence_level * 100, 'first_name': first, # ... }
  69. Things You Should NOT Do

  70. Stop At A Low Level “This Low Level API ought

    to be good enough.” Yeah, right. And 640Kb was totally enough for everyone.
  71. Stop At A Low Level def get_page(socket, timeout): # Use

    the open socket to fetch # the web page. It's a open # socket to the correct place, # right?
  72. Stop At A Low Level def get_raw_page(socket, timeout=60): # Do

    the actual fetch. return headers, content def get_page(url, timeout=None, fetcher=get_raw_page): # Validate the URL. # Open a socket. head, content = fetcher(my_socket) if head.status == 500: raise OhNoes('...') # ...
  73. Wildly Different Return Values “Hm, does this return an integer

    or a dictionary?”
  74. Wildly Different Return Values def query(engine_url, query, limit=10): # Make

    the user deal with the raw data. results = [res for res in results if res] return results, total def raw_query(engine_url, query, limit=10): # Make the user deal with the raw data. return { 'search_results': results, 'count': total, }
  75. Wildly Different Return Values def query(engine_url, query, limit=10): # We're

    doing a search query. # Return sane, similar structures. return { 'results': results, 'total': total_count, } def raw_query(engine_url, query, limit=10): # We're doing a search query. # Return sane, similar structures. return { 'results': results, 'total': total_count, }
  76. Useless “Implementation” Code If it can’t do anything for itself,

    it’s code without purpose.
  77. Useless “Implementation” Code class BaseBackend(object): # Full implementation. def __init__(self,

    url): raise NotImplementedError('...') def save(self, filename, data): raise NotImplementedError('...') def load(self, filename): raise NotImplementedError('...')
  78. Useless “Implementation” Code class FileBackend(object): # Does useful things. Subclasses

    can # override less. def __init__(self, url): self.url = url def save(self, filename, data): the_file = open(filename, 'w') return json.dump(the_file, data) def load(self, filename): data = open(filename) return json.load(data)
  79. If It’s Difficult To Test... ...you screwed up. But maybe

    you can fix it.
  80. If It’s Difficult To Test... def current_temperature(station): # Oh yeah,

    this totally works BTW. url = 'http://w1.weather.gov/obhistory/' + \ '%s.html' % station resp = requests.get(url) data = pq(resp.content) return float(data('table').eq(3)\ .find('tr').eq(4)\ .find('td').eq(6).text())
  81. If It’s Difficult To Test... class Weather(object): def build_url(self, station):

    return 'http://w1.weather.gov/obhistory/' + \ '%s.html' % station def fetch_page(self, url): return requests.get(url).content def parse_page(self, content): return pq(resp.content) def last_conditions(self, data): return data('table').eq(3).find('tr').eq(4) def current_temperature(self, station): url = self.build_url(station) content = self.fetch_page(url) data = self.parse_page(content) return float(self.last_conditions(data)\ .find('td').eq(6).text())
  82. django-Specific Topics

  83. Pluggable Backend ALL THE THINGS

  84. Internationalize ALL THE THINGS

  85. Dynamically Loadable Classes/Code 60% more error-handling, every time.

  86. Declarative Syntax Metaclasses: Call your doctor if headaches last more

    than 4 hours.
  87. Don’t metaclass ALL THE THINGS

  88. The ORM Love it or hate it, there’s some great

    learning opportunities there to be had.
  89. Other Ideas

  90. Very Little Global State # Only defaults or things that

    absolute # have to be module-level. BASE_URL = 'http://w1.weather.gov/obhistory/' TIMEOUT = 10 # Make them defaults but allow passing # different in their place. def fetch_page(self, url=BASE_URL, timeout=TIMEOUT): return requests.get( url, timeout=timeout ).content
  91. Decrease Reliance On ``self`` class Weather(object): def build_url(self, station): #

    Make a URL. def fetch_page(self, url): # Get a page def parse_page(self, content): # Use the page content. def current_temperature(self, station): url = self.build_url(station) content = self.fetch_page(url) data = self.parse_page(content) # You could be using ``self.<variable>`` on # these, but by allowing data to be passed # in, you make it easier to test.
  92. Resist The Urge To Use Magic Be explicit first, then

    add shortcuts. (which can be a little more magical)
  93. So...

  94. So... •Use the Golden Rule.

  95. So... •Use the Golden Rule. •Consistency is key.

  96. So... •Use the Golden Rule. •Consistency is key. •Plan for

    the worst & include sweet shortcuts for the best.
  97. So... •Use the Golden Rule. •Consistency is key. •Plan for

    the worst & include sweet shortcuts for the best. •Write a thing you’d to use...
  98. So... •Use the Golden Rule. •Consistency is key. •Plan for

    the worst & include sweet shortcuts for the best. •Write a thing you’d to use... •...then make it even better.
  99. Thanks!

  100. I’m Daniel Lindsley of Toast Driven @toastdriven http://toastdriven.com/