Consuming APIs: reporting from the trenches (PHPBenelux Conference 2019)

Integrating multiple API endpoints into a single application can be challenging. In this talk I will go over a lot of problems and how the can be solved. Going from easy authentication to locally caching calls via middlewares and using webhooks (or callbacks) for notification, this talk covers it all (or at least tries to).

Transcript

1. Consuming APIs
   Consuming APIs
   Consuming APIs
   Consuming APIs
   Consuming APIs
   Consuming APIs
   Consuming APIs
   Consuming APIs
   reporting from the trenches
   reporting from the trenches
   reporting from the trenches
   reporting from the trenches
   reporting from the trenches
   reporting from the trenches
   reporting from the trenches
   reporting from the trenches
   Jachim Coudenys
   Jachim Coudenys
   Jachim Coudenys
   Jachim Coudenys
   Jachim Coudenys
   Jachim Coudenys
   Jachim Coudenys
   Jachim Coudenys
   jachim.coudenys@combell.group
   jachim.coudenys@combell.group
   jachim.coudenys@combell.group
   jachim.coudenys@combell.group
   jachim.coudenys@combell.group
   jachim.coudenys@combell.group
   jachim.coudenys@combell.group
   jachim.coudenys@combell.group
   @coudenysj
   @coudenysj
   @coudenysj
   @coudenysj
   @coudenysj
   @coudenysj
   @coudenysj
   @coudenysj
   

   View Slide

2. APIs?
   Web API
   Request - Response
   Content-Type: *
   SOAP
   

   View Slide

3. file_get_contents()
   fopen()
   

   View Slide

4. $response = json_decode(
   file_get_contents(
   'https://api.meetup.com/php-wvl'
   )
   );
   dump($response);
   

   View Slide

5. {#3
   +"id": 12568822
   +"name": "PHP-WVL"
   +"status": "active"
   +"link": "https://www.meetup.com/php-wvl/"
   +"urlname": "php-wvl"
   +"description": "A PHP Usergroup for all PHP devs based in West-Vlaanderen."
   +"created": 1390997367000
   +"city": "Torhout"
   +"untranslated_city": "Torhout"
   +"country": "BE"
   +"localized_country_name": "Belgium"
   +"localized_location": "Torhout, Belgium"
   +"state": ""
   +"join_mode": "open"
   +"visibility": "public"
   +"lat": 51.07
   +"lon": 3.1
   +"members": 427
   +"organizer": {#2
   +"id": 84915262
   

   View Slide

6. $response = json_decode(
   file_get_contents(
   'https://api.meetup.com/mqskjdfqsdjjfaoijqmodiffqsjdf'
   )
   );
   dump($response);
   

   View Slide

7. failed to open stream: HTTP request failed! HTTP/1.1
   404 Not Found
   PHP Warning: file_get_contents(https://api.meetup.com/mqskjdfqsdjjfaoijqmodiffqsjdf): failed to open stream
   in 01/2_file_get_contents_exceptions.php on line 9
   PHP Stack trace:
   PHP 1. {main}() 01/2_file_get_contents_exceptions.php:0
   PHP 2. file_get_contents() 01/2_file_get_contents_exceptions.php:9
   PHP Fatal error: Uncaught TypeError: json_decode() expects parameter 1 to be string, boolean given in 01/
   Stack trace:
   #0 01/2_file_get_contents_exceptions.php(9): json_decode(false)
   #1 {main}
   thrown in 01/2_file_get_contents_exceptions.php on line 9
   

   View Slide

8. allow_url_fopen
   This option enables the URL-aware
   fopen wrappers that enable accessing
   URL object like files. Default wrappers
   are provided for the access of remote
   files using the ftp or http protocol,
   some extensions like zlib may register
   additional wrappers.
   

   View Slide

9. We need something more robust
   

   View Slide

10. Jachim Coudenys
    

    View Slide

11. View Slide

12. View Slide

13. Consuming APIs
    Consuming APIs
    Consuming APIs
    Consuming APIs
    Consuming APIs
    Consuming APIs
    Consuming APIs
    Consuming APIs
    reporting from the trenches
    reporting from the trenches
    reporting from the trenches
    reporting from the trenches
    reporting from the trenches
    reporting from the trenches
    reporting from the trenches
    reporting from the trenches
    

    View Slide

14. Combell Internal
    Project
    Evolution in project
    Started small, escalated quickly
    Average of 120 API calls per request
    

    View Slide

15. guzzlehttp/guzzle
    

    View Slide

16. Guzzle is a PHP HTTP client that
    makes it easy to send HTTP requests
    and trivial to integrate with web
    services.
    

    View Slide

17. Simple interface for building query strings, POST
    requests, streaming large uploads, streaming
    large downloads, using HTTP cookies, uploading
    JSON data, etc...
    Synchronous and asynchronous requests.
    Uses PSR-7 interfaces for requests, responses,
    and streams. This allows you to utilize other PSR-
    7 compatible libraries with Guzzle.
    No hard dependency on cURL, PHP streams,
    sockets, or non-blocking event loops.
    Middleware system allows you to augment and
    compose client behavior.
    

    View Slide

18. Guzzle basics
    Define request headers
    Inspect response headers
    Stream body
    Exception handling
    

    View Slide

19. php 02/*.php
    

    View Slide

20. Guzzle abstracted
    Connection class
    Abstract away specific calls
    Add multiple API endpoints
    

    View Slide

21. php 03/*.php
    

    View Slide

22. logic
    

    View Slide

23. Authentication
    Every API has its own way of auth
    Most of them use Authorization header
    getToken() in subclasses
    

    View Slide

24. php 04/*.php
    

    View Slide

25. Traceability
    We have a 'generic' request method
    Let's add logging
    And keep track of performance
    

    View Slide

26. php 05/*.php
    

    View Slide

27. Middleware
    

    View Slide

28. https://stackphp.com/img/onion.png
    

    View Slide

29. HTTP message interfaces
    Request messages
    Response messages
    Headers
    PSR-7
    

    View Slide

30. Server
    use Tari\ServerMiddlewareInterface;
    use Tari\ServerFrameInterface;
    use Psr\Http\Message\ServerRequestInterface;
    use Psr\Http\Message\ResponseInterface;
    class Foo implements ServerMiddlewareInterface
    {
    public function handle(
    ServerRequestInterface $request,
    ServerFrameInterface $frame
    ): ResponseInterface
    {
    if ($this->isBadRequest($request)) {
    return $frame->factory()
    ->createResponse("Bad Request", 400);
    }
    return $frame->next($request);
    

    View Slide

31. Server
    A/B Testing
    Debugging
    Caching
    CORS
    CSRF Protection
    HTTP Basic Auth
    OAuth 2.0
    OpenID
    Rate Limiting
    Referrals
    IP Restriction
    

    View Slide

32. Client
    function (callable $handler) {
    return function (RequestInterface $request, array $options)
    use ($handler)
    {
    return $handler($request, $options);
    };
    }
    

    View Slide

33. StackPHP
    

    View Slide

34. HTTP Server Request Handlers
    PSR-15
    

    View Slide

35. Guzzle Handlers and
    Middleware
    

    View Slide

36. php 06/*.php
    

    View Slide

37. GuzzleHttp\Middleware
    

    View Slide

38. https://www.google.be/search?
    q=guzzle%20middleware
    

    View Slide

39. Local caching layer
    and
    rfc7234 Cache Docs
    

    View Slide

40. https://www.sitepoint.com/getting-started-with-varnish/
    

    View Slide

41. Local caching layer
    ModerateCacheStrategy *
    kevinrob/guzzle-cache-middleware
    PrivateCacheStrategy
    PublicCacheStrategy
    GreedyCacheStrategy
    

    View Slide

42. \GuzzleHttp\Middleware
    ::mapResponse
    Add mapResponse middleware
    (so caching works as expected)
    200 -> 404
    403
    500 -> 404
    etc...
    

    View Slide

43. Automatically retries HTTP requests when a server responds with a 429
    or 503 status
    Sets a retry delay based on the Retry-After HTTP header, if it is sent, or
    automatically backs off exponentially if no Retry-After header is sent
    Optionally retries requests that time out (based on the connect_timeout
    or timeout options)
    Set an optional callback when a retry occurs (useful for
    logging/reporting)
    Specify a maximum number of retry attempts before giving up (default:
    10)
    caseyamcl/ guzzle_retry_middleware
    

    View Slide

44. \GuzzleHttp\Middleware
    ::mapResponse
    Add mapResponse middleware
    (so retry works as expected)
    500 -> 429
    500 -> 503
    etc...
    

    View Slide

45. Webhooks / Callbacks
    Don't poll
    Register callback url
    (https://app.local/callback/34dl2F4v)
    https://swagger.io/docs/specification/callbacks/
    

    View Slide

46. @coudenysj
    jachim.coudenys@combell.group
    Thank you
    https://joind.in/talk/f71d9
    

    View Slide