Tweet
Tweet

Slide 1

Slide 1 text

High performing API with HTTP/2

Slide 2

Slide 2 text

Julien Viet Open source developer for 15+ years Current @vertx_project lead Principal software engineer at Marseille Java User Group Leader https://www.julienviet.com/ http://github.com/vietj @julienviet  https://www.mixcloud.com/cooperdbi/

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

Page Load Time as bandwidth increases 1000 1550 2100 2650 3200 1Mbps 2Mbps 2Mbps 4Mbps 5Mbps 6Mbps 7Mbps 8Mbps 9Mbps10Mbps Page Load Time as latency decrease 1000 1750 2500 3250 4000 200 ms 180 ms 160 ms 140 ms 120 ms 100 ms 80 ms 60 ms 40 ms 20 ms

Slide 6

Slide 6 text

Web performance state of the art

Slide 7

Slide 7 text

Persistent connections

Slide 8

Slide 8 text

Multiple connections

Slide 9

Slide 9 text

Domain sharding

Slide 10

Slide 10 text

Content Delivery Networks

Slide 11

Slide 11 text

Caching

Slide 12

Slide 12 text

Compression and minification

Slide 13

Slide 13 text

Still a bottleneck! GET OK GET OK

Slide 14

Slide 14 text

HTTP/2 A better TCP transport for HTTP request and responses Same HTTP semantics RFC7540 : Hypertext Protocol version 2 RFC7541 : Header Compression for HTTP/2

Slide 15

Slide 15 text

HTTP/2 H2 TLS/SSL Mandatory for browsers ALPN extension to negotiate HTTP/1 or HTTP/2 H2C Clear text Via HTTP/1 upgrade or directly

Slide 16

Slide 16 text

HTTP/2 framed protocol Defines a set of frames encoded in binary on a single connection Settings Headers Data Flow control Push, Priority, Ping Reset

Slide 17

Slide 17 text

Settings frames First frame exchanged by client and server Max concurrency Max frame size etc…

Slide 18

Slide 18 text

Request headers GET /index.html HTTP/1.1\r\n Host: !!www.free.fr\r\n User-Agent: Mozilla/5.0\r\n Accept-Encoding: text/html\r\n Accept-Language: en-US\r\n \r\n 00003501250000000300 0000000f824488355217 caf3a69a3f874189f1e3 c2f2d852af2d9f7a88d0 7f66a281b0dae0508749 7ca589d34d1f51842d4b 70dd length type flags stream_id 2x smaller

Slide 19

Slide 19 text

Subsequent request headers GET /products.html HTTP/1.1\r\n Host: !!www.free.fr\r\n User-Agent: Mozilla/5.0\r\n Accept-Encoding: text/html\r\n Accept-Language: en-US\r\n \r\n 00001701250000000500 0000000f82448aaec3c9 691285e74d347f87c2c1 c0bf 4x smaller

Slide 20

Slide 20 text

Response headers + data HTTP/1.1 200 OK\r\n Content-Type: text/html\r\n Cache-Control: max-age=86400\r\n Content-Length: 37\r\n \r\n Hello World !! 0000250001000000033c 68746d6c3e3c626f6479 3e48656c6c6f20576f72 6c643c2f626f64793e3c 2f68746d6c3e 00001f01240000000300 0000000f885f87497ca5 89d34d1f588aa47e561c c581e71a003f5c023337 length type flags stream_id

Slide 21

Slide 21 text

DEMO

Slide 22

Slide 22 text

HTTP/2 concurrency works!

Slide 23

Slide 23 text

Scalable microservices with HTTP/2

Slide 24

Slide 24 text

Target architectures service service gateway backend service backend

Slide 25

Slide 25 text

What is performance ? Performance is hard to define but in this talk we will study how HTTP/2 protocol is a better transport for scaling microservices

Slide 26

Slide 26 text

Context 0.1 ms < ping <1 ms 1 G < bandwidth < 10 G 1 ms < service time < 100 ms 100 b < body size < 10 kb

Slide 27

Slide 27 text

HTTP/1 vs HTTP/2 server HTTP/1 vs HTTP/2 backend client 20 ms think time

Slide 28

Slide 28 text

performed (req/sec) 0 200 400 600 800 planned (req / sec) 0 100 200 300 400 500 600 700 800 HTTP/1 - 8 connections

Slide 29

Slide 29 text

HTTP/1.1 Client Server }20ms Backend GET response request 200 OK }

Slide 30

Slide 30 text

performed (req / sec) 0 300 600 900 1200 planned (req / sec) 0 100 200 300 400 500 600 700 800 900 1000 1100 1200 HTTP/1 - 8 connections HTTP/2 - 1 connection / max_concurrent_streams 20

Slide 31

Slide 31 text

HTTP/2 multiplexing Client Server 20ms Backend GET response request 200 OK } } }

Slide 32

Slide 32 text

Capacity planning Configure MAX_CONCURRENT_STREAM Backend connection sizing Avoid thread pools

Slide 33

Slide 33 text

Thread pool concurrency Backend Workers HTTP/2 connection

Slide 34

Slide 34 text

Thread pool concurrency Backend Workers HTTP/2 connection

Slide 35

Slide 35 text

Thread pool concurrency Backend Workers HTTP/2 connection

Slide 36

Slide 36 text

Thread pool concurrency Backend Workers HTTP/2 connection

Slide 37

Slide 37 text

Thread pool concurrency Backend Workers HTTP/2 connection

Slide 38

Slide 38 text

performed (req / sec) 0 1250 2500 3750 5000 planned (req / sec) 0 1000 2000 3000 4000 5000 HTTP/2 thread pool

Slide 39

Slide 39 text

Eclipse Vert.x Open source project started in 2012 Created by Tim Fox Eclipse / Apache licensing A toolkit for building reactive applications for the JVM 7K ⋆ on Built on top of https://vertx.io @vertx_project

Slide 40

Slide 40 text

Vert.x Latest and greatest Vert.x 3.4.2 Scala and Kotlin support RxJava improvements MQTT server Kafka client gRPC support Web client Infinispan cluster manager and much more!

Slide 41

Slide 41 text

Toolkit Embeddable Composable Modular Minimum dependencies Classloading / Injection free

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

Reactive Non blocking Event driven Distributed Rxified APIs Reactive-streams

Slide 44

Slide 44 text

HTTP/2 with Vert.x Client / Server h2 / h2c HTTP Request / response API HTTP/2 specific features for extensions

Slide 45

Slide 45 text

Event driven Ȑ NIO selectors disk operations timers messages  database etc…

Slide 46

Slide 46 text

Reactor pattern with Event Loop Single thread

Slide 47

Slide 47 text

Events at scale Ȑ      Ȑ Ȑ Ȑ Ȑ Ȑ buffer!->{…} timerID!->{…} asyncFile!->{…} rows!->{…} message!->{…}

Slide 48

Slide 48 text

Non-blocking IO benefits ✓ Handle many connections with a few threads - focus on protocol concurrency - minimize system calls - more efficient for pipelined/multiplexed protocols ✓ Get away from thread pools - removes unnecessary bottlenecks - easier capacity planning - focus on protocol concurrency ✓ Gracefully handle slow connection - remain responsive - don’t impact other connections

Slide 49

Slide 49 text

Thread pool concurrency Backend Event Loop HTTP/2 connection

Slide 50

Slide 50 text

performed (req / sec) 0 1250 2500 3750 5000 planned (req / sec) 0 1000 2000 3000 4000 5000 HTTP/2 thread pool HTTP/2 non blocking

Slide 51

Slide 51 text

Going multicore

Slide 52

Slide 52 text

Multi-reactor pattern

Slide 53

Slide 53 text

performed (req / sec) 0 3250 6500 9750 13000 planned (req / sec) 0 1000 3000 5000 7000 9000 11000 13000 HTTP/2 blocking HTTP/2 non blocking -1 core HTTP/2 non blocking - 2 cores

Slide 54

Slide 54 text

REACTIVE ECOSYSTEM

Slide 55

Slide 55 text

Building Reactive Microservices in Java https://developers.redhat.com/promotions/building-reactive-microservices-in-java/

Slide 56

Slide 56 text

TL;DR; Unleash concurrency with HTTP/2 and gRPC Non blocking is a key factor for high concurrency Vert.x is a great fit for HTTP/2 Reactive ecosystem Easy to scale

Slide 57

Slide 57 text

Merci!!!