API Tips From the Frontline

Starting to write an API is an easy task, but you quickly stumble upon many obstacles and hard decisions. How to manage result pagination? How to handle write operations, file uploads and authentication? Join me as I share my tricks that allowed me to ship high-profile projects in record time while keeping the code clean and maintainable.

Transcript

1. foolab.ca | @foolabca
   API Tips From the
   Frontline
   CakeFest, Amsterdam - May 29, 2016
   

   View Slide

2. What you will learn
   • Don't repeat my mistakes.
   • Overcome common obstacles.
   • Build elegant & pragmatic
   APIs.
   2
   

   View Slide

3. Anna Filina
   • Developer
   • Problem solver
   • Teacher
   • Advisor
   • FooLab + ConFoo
   3
   

   View Slide

4. Endpoints
   What URL to call when
   

   View Slide

5. Lists
   • /products
   • /products?category=games,movies&lang=fr
   • /products?include=platforms
   • /products?sort=-date
   • /products?pageNumber=2&pageSize=100
   • /products?lang=fr
   5
   

   View Slide

6. Details
   • /products/1
   • /products/1?include=photos,reviews
   6
   

   View Slide

7. Response example: list
   {
   "data": [
   {"name": "Skyrim"},
   {"name": "Civilization V"}
   ],
   "meta": {
   "pages": 50,
   "count": 100
   }
   }
   7
   

   View Slide

8. Write
   • POST /products
   • PUT or PATCH /products/1
   • DELETE /products/1
   • Version your APIs:
   ◦ api.example.org/v1/products
   ◦ Headers also an option.
   8
   

   View Slide

9. Request/Response
   How to format stuff
   

   View Slide

10. Multiple formats
    • xml, json, html, etc.
    • Can use 1 endpoint with Accept header.
    • Can send version, pagination & language info using
    headers too.
    10
    

    View Slide

11. Request
    • Bad: Content-Type: application/x-www-form-urlencoded
    • Content-Type: application/json
    • Send content (POST) in body, not headers.
    11
    

    View Slide

12. Request example: add
    POST /products HTTP/1.1
    Host: api.example.org
    Content-Type: application/json; charset=UTF-8
    {
    "data": {
    "name": "Overwatch",
    "price": 49.99
    }
    }
    12
    

    View Slide

13. 3 queries
    • Count query with same filters but no pagination.
    • Query with filters, pagination & sorting, but only select
    DISTINCT IDs
    ◦ Because JOIN + LIMIT = wrong number of root records.
    • Query all the data (with joins) by IDs and apply sorting.
    13
    

    View Slide

14. LIMIT
    SELECT * FROM game
    LEFT JOIN rating ON rating.game_id = game.id
    WHERE game.price < 20
    LIMIT 2;
    14
    id name price game_id score
    1 Skyrim 19.99 1 9.1
    1 Skyrim 19.99 1 8.9
    2 Diablo 3 17.89 2 8.5
    

    View Slide

15. Upload
    • Use tools, don't DIY (do it yourself).
    • Client-side: Plupload
    ◦ Step 1: upload temp file.
    ◦ Step 2: give file path to next API request.
    • Server-side: Guzzle.
    15
    

    View Slide

16. Plupload
    var uploader = new plupload.Uploader({
    runtimes: 'html5,html4',
    max_file_size: '5mb',
    url: '/api/upload',
    filters: [{extensions: 'jpg,png,jpeg'}]
    });
    uploader.init();
    16
    

    View Slide

17. Guzzle
    $url = 'http://example.org/profiles/1/edit';
    $request = $client->createRequest('PATCH', $url);
    $reqBody = $request->getBody();
    $reqBody->setField('data', ['first_name':'Anna']);
    $file = new PostFile('i.jpg', fopen('/path', 'r'));
    $reqBody->addFile($file);
    $response = $client->send($request);
    17
    

    View Slide

18. Status codes
    • Don't confuse API & HTTP codes.
    • 2xx success.
    • 3xx redirect.
    • 4xx client error.
    • 5xx server error.
    • Send API-specific code in body.
    18
    

    View Slide

19. Example of error
    Status: 400
    Content-Type: application/json; charset=UTF-8
    {
    "errors": {[
    "code": 1001,
    "message": "Price must be greater than 0."
    ]}
    }
    19
    

    View Slide

20. Testing
    Simpler than you think
    

    View Slide

21. HTTP
    • Use Guzzle or built-in framework tools.
    • Generate HTTP request , compare output.
    21
    

    View Slide

22. Guzzle test
    // tests/ApiProductTest.php
    public function test_GetOneProduct_ReturnsSuccess()
    {
    $client = new Client();
    $response = $client->get('http://example.org/products/1', [
    'exceptions' => false,
    'headers' => ['Accept' => 'application/json']
    ]);
    $this->assertEquals(200, $response->getStatusCode());
    // ...
    }
    22
    

    View Slide

23. Guzzle test
    // ...
    $body = $response->getBody()->getContents();
    $this->assertJsonStringEqualsJsonString('{
    "data": {
    "id": "1",
    "name": "Overwatch",
    "price": 49.99
    }
    }', $body);
    23
    

    View Slide

24. Testing tips
    • Create separate database for tests.
    • Write tests before you code:
    ◦ TDD.
    ◦ Contract between you and client dev.
    24
    

    View Slide

25. Authentication
    Nooo, I hate that part! Someone else code it plz.
    

    View Slide

26. Multiple methods
    • Don't send username/password in each request.
    ◦ Especially with untrusted 3rd parties.
    ◦ Especially if no SSL.
    • You can have multiple auth methods for one API.
    • Sessions similar to tokens.
    26
    

    View Slide

27. OAuth2
    • SSL required (can be risky).
    • Advanced features like access scope.
    • Can be overkill if you need basic features.
    • Private credentials.
    27
    

    View Slide

28. OAuth2 - conceptual diagram
    28
    User Client app

    (php/js/mobile)
    API
    Request Forward
    Validate
    Create
    token
    Store
    token
    Login form
    Login
    token
    user/pass
    

    View Slide

29. Digest
    • Its own encryption.
    • Easy to implement.
    • No replay (nonce).
    • Comes out-of-the-box with some frameworks.
    29
    

    View Slide

30. Digest - conceptual diagram
    30
    Request
    User/client app API
    Unauthorized
    Validate
    digest (nonce,pass)
    Request
    nonce
    

    View Slide

31. Refactoring to API
    Don't rewrite all the legacy at once
    

    View Slide

32. Progressive rewrite
    • Rewrite one component/module at a time.
    • Start with whatever has fewer dependencies (or critical).
    • Delete dead code.
    • Copy production data for dev environment.
    32
    

    View Slide

33. Services vs libraries
    • Services: GET /products?id=1,5
    • Libraries /classes:
    • Bypass HTTP
    • $table->getApiList( ["id" => "1,5"] );
    • Can fetch without pagination.
    • Can hydrate to models.
    33
    

    View Slide

34. Implementation tips
    Is this a good way to code it?
    

    View Slide

35. Libraries
    • I use MVC frameworks & ORMs.
    • Aside from frameworks, I prefer small tools that don't do
    too much magic.
    35
    

    View Slide

36. Reuse & standardize
    • Goal: streamline endpoint creation.
    • Automate request parameters extraction.
    • Automate query generation based on parameters.
    • Keep things fully customizable.
    • Decouple from framework.
    36
    

    View Slide

37. Performance
    Make things faster. Much faster.
    

    View Slide

38. Benchmark
    • Give Tideways or Blackfire a spin.
    • Make performance part of your test suite.
    • In dev/test mode, use a meta block.
    38
    

    View Slide

39. Example
    {
    "meta": {
    "perf": {
    "db_time": 0.00367,
    "total_time": 0.3120,
    "memory": 19661
    }
    }
    }
    39
    

    View Slide

40. Performance tips
    • Don't use lazy loading. Example: $product->getPhotos().
    • Craft your own joins and carefully select fields.
    • Avoid ORM built-in hydration for read operations.
    40
    

    View Slide

41. Performance tips
    • Use API keys even for public endpoints (DDoS
    mitigation).
    • Put stuff in Memcached/Redis (especially blocked keys).
    • HTTP server can check headers.
    41
    

    View Slide

42. Useful links
    • Standard API format

    http://jsonapi.org/format/
    • Digest implementation

    http://php.net/manual/en/features.http-auth.php
    • Book "Build APIs You Won't Hate"

    https://leanpub.com/build-apis-you-wont-hate
    42
    

    View Slide

43. Anna Filina
    • Development.
    • Fix bugs & performance issues.
    • Workshops on testing, frameworks & APIs.
    • Advisor on testing strategy, legacy code.
    43
    

    View Slide

44. @afilina afilina.com
    joind.in
    

    View Slide