Upgrade to Pro — share decks privately, control downloads, hide ads and more …

High performing API with HTTP/2

High performing API with HTTP/2

Julien Viet

October 20, 2017
Tweet

More Decks by Julien Viet

Other Decks in Programming

Transcript

  1. High performing API with
    HTTP/2

    View full-size slide

  2. Julien Viet
    Open source developer for 15+ years
    Current @vertx_project lead
    Principal software engineer at
    Marseille Java User Group Leader
    https://www.julienviet.com/
    http://github.com/vietj
    @julienviet
     https://www.mixcloud.com/cooperdbi/

    View full-size slide

  3. Page Load Time as bandwidth increases
    1000
    1550
    2100
    2650
    3200
    1Mbps 2Mbps 2Mbps 4Mbps 5Mbps 6Mbps 7Mbps 8Mbps 9Mbps10Mbps
    Page Load Time as latency decrease
    1000
    1750
    2500
    3250
    4000
    200 ms 180 ms 160 ms 140 ms 120 ms 100 ms 80 ms 60 ms 40 ms 20 ms

    View full-size slide

  4. Web performance state of the art

    View full-size slide

  5. Persistent connections

    View full-size slide

  6. Multiple connections

    View full-size slide

  7. Domain sharding

    View full-size slide

  8. Content Delivery Networks

    View full-size slide

  9. Compression and minification

    View full-size slide

  10. Still a bottleneck!
    GET
    OK
    GET
    OK

    View full-size slide

  11. HTTP/2
    A better TCP transport for HTTP request and responses
    Same HTTP semantics
    RFC7540 : Hypertext Protocol version 2
    RFC7541 : Header Compression for HTTP/2

    View full-size slide

  12. HTTP/2
    H2
    TLS/SSL
    Mandatory for browsers
    ALPN extension to negotiate HTTP/1 or HTTP/2
    H2C
    Clear text
    Via HTTP/1 upgrade or directly

    View full-size slide

  13. HTTP/2 framed protocol
    Defines a set of frames encoded in binary on a single
    connection
    Settings
    Headers
    Data
    Flow control
    Push, Priority, Ping
    Reset

    View full-size slide

  14. Settings frames
    First frame exchanged by client and server
    Max concurrency
    Max frame size
    etc…

    View full-size slide

  15. Request headers
    GET /index.html HTTP/1.1\r\n
    Host: !!www.free.fr\r\n
    User-Agent: Mozilla/5.0\r\n
    Accept-Encoding: text/html\r\n
    Accept-Language: en-US\r\n
    \r\n
    00003501250000000300
    0000000f824488355217
    caf3a69a3f874189f1e3
    c2f2d852af2d9f7a88d0
    7f66a281b0dae0508749
    7ca589d34d1f51842d4b
    70dd
    length type flags stream_id
    2x smaller

    View full-size slide

  16. Subsequent request headers
    GET /products.html HTTP/1.1\r\n
    Host: !!www.free.fr\r\n
    User-Agent: Mozilla/5.0\r\n
    Accept-Encoding: text/html\r\n
    Accept-Language: en-US\r\n
    \r\n
    00001701250000000500
    0000000f82448aaec3c9
    691285e74d347f87c2c1
    c0bf
    4x smaller

    View full-size slide

  17. Response headers + data
    HTTP/1.1 200 OK\r\n
    Content-Type: text/html\r\n
    Cache-Control: max-age=86400\r\n
    Content-Length: 37\r\n
    \r\n
    Hello World
    !!
    0000250001000000033c
    68746d6c3e3c626f6479
    3e48656c6c6f20576f72
    6c643c2f626f64793e3c
    2f68746d6c3e
    00001f01240000000300
    0000000f885f87497ca5
    89d34d1f588aa47e561c
    c581e71a003f5c023337
    length type flags stream_id

    View full-size slide

  18. HTTP/2
    concurrency
    works!

    View full-size slide

  19. Scalable microservices with HTTP/2

    View full-size slide

  20. Target architectures
    service
    service
    gateway
    backend
    service
    backend

    View full-size slide

  21. What is performance ?
    Performance is hard to define but in this talk we will study
    how HTTP/2 protocol is a better transport for scaling
    microservices

    View full-size slide

  22. Context
    0.1 ms < ping <1 ms
    1 G < bandwidth < 10 G
    1 ms < service time < 100 ms
    100 b < body size < 10 kb

    View full-size slide

  23. HTTP/1 vs HTTP/2
    server
    HTTP/1 vs HTTP/2
    backend
    client
    20 ms
    think time

    View full-size slide

  24. performed (req/sec)
    0
    200
    400
    600
    800
    planned (req / sec)
    0 100 200 300 400 500 600 700 800
    HTTP/1 - 8 connections

    View full-size slide

  25. HTTP/1.1
    Client Server
    }20ms
    Backend
    GET
    response
    request
    200 OK
    }

    View full-size slide

  26. performed (req / sec)
    0
    300
    600
    900
    1200
    planned (req / sec)
    0 100 200 300 400 500 600 700 800 900 1000 1100 1200
    HTTP/1 - 8 connections
    HTTP/2 - 1 connection / max_concurrent_streams 20

    View full-size slide

  27. HTTP/2 multiplexing
    Client Server
    20ms
    Backend
    GET
    response
    request
    200 OK
    }
    }
    }

    View full-size slide

  28. Capacity planning
    Configure MAX_CONCURRENT_STREAM
    Backend connection sizing
    Avoid thread pools

    View full-size slide

  29. Thread pool concurrency
    Backend
    Workers
    HTTP/2 connection

    View full-size slide

  30. Thread pool concurrency
    Backend
    Workers
    HTTP/2 connection

    View full-size slide

  31. Thread pool concurrency
    Backend
    Workers
    HTTP/2 connection

    View full-size slide

  32. Thread pool concurrency
    Backend
    Workers
    HTTP/2 connection

    View full-size slide

  33. Thread pool concurrency
    Backend
    Workers
    HTTP/2 connection

    View full-size slide

  34. performed (req / sec)
    0
    1250
    2500
    3750
    5000
    planned (req / sec)
    0 1000 2000 3000 4000 5000
    HTTP/2 thread pool

    View full-size slide

  35. Eclipse Vert.x
    Open source project started in 2012
    Created by Tim Fox
    Eclipse / Apache licensing
    A toolkit for building reactive applications for the JVM
    7K ⋆ on
    Built on top of
    https://vertx.io
    @vertx_project

    View full-size slide

  36. Vert.x
    Latest and greatest Vert.x 3.4.2
    Scala and Kotlin support
    RxJava improvements
    MQTT server
    Kafka client
    gRPC support
    Web client
    Infinispan cluster manager
    and much more!

    View full-size slide

  37. Toolkit
    Embeddable
    Composable
    Modular
    Minimum dependencies
    Classloading / Injection free

    View full-size slide

  38. Reactive
    Non blocking
    Event driven
    Distributed
    Rxified APIs
    Reactive-streams

    View full-size slide

  39. HTTP/2 with Vert.x
    Client / Server
    h2 / h2c
    HTTP Request / response API
    HTTP/2 specific features for extensions

    View full-size slide

  40. Event driven
    Ȑ NIO selectors
    disk operations
    timers
    messages
     database
    etc…

    View full-size slide

  41. Reactor pattern with Event Loop
    Single
    thread

    View full-size slide

  42. Events at scale
    Ȑ





    Ȑ
    Ȑ
    Ȑ
    Ȑ
    Ȑ
    buffer!->{…}
    timerID!->{…}
    asyncFile!->{…}
    rows!->{…}
    message!->{…}

    View full-size slide

  43. Non-blocking IO benefits
    ✓ Handle many connections with a few threads
    - focus on protocol concurrency
    - minimize system calls
    - more efficient for pipelined/multiplexed protocols
    ✓ Get away from thread pools
    - removes unnecessary bottlenecks
    - easier capacity planning
    - focus on protocol concurrency
    ✓ Gracefully handle slow connection
    - remain responsive
    - don’t impact other connections

    View full-size slide

  44. Thread pool concurrency
    Backend
    Event Loop
    HTTP/2 connection

    View full-size slide

  45. performed (req / sec)
    0
    1250
    2500
    3750
    5000
    planned (req / sec)
    0 1000 2000 3000 4000 5000
    HTTP/2 thread pool HTTP/2 non blocking

    View full-size slide

  46. Going multicore

    View full-size slide

  47. Multi-reactor pattern

    View full-size slide

  48. performed (req / sec)
    0
    3250
    6500
    9750
    13000
    planned (req / sec)
    0 1000 3000 5000 7000 9000 11000 13000
    HTTP/2 blocking
    HTTP/2 non blocking -1 core
    HTTP/2 non blocking - 2 cores

    View full-size slide

  49. REACTIVE ECOSYSTEM

    View full-size slide

  50. Building Reactive Microservices in Java
    https://developers.redhat.com/promotions/building-reactive-microservices-in-java/

    View full-size slide

  51. TL;DR;
    Unleash concurrency with HTTP/2 and gRPC
    Non blocking is a key factor for high concurrency
    Vert.x is a great fit for HTTP/2
    Reactive ecosystem
    Easy to scale

    View full-size slide