Vert.x : un écosystème pour construire des applications réactives de toutes tailles

Slide 1

Slide 1 text

Vert.x : un écosystème pour construire des applications réactives de toutes tailles Julien Ponge

Slide 2

Slide 2 text

Julien Ponge Maitre de Conférences “Delegated consultant to Red Hat” on Vert.x Eclipse Golo + extensive F/OSS background ! https://julien.ponge.org/ " @jponge # @jponge  https://www.mixcloud.com/hclcast/

Slide 3

Slide 3 text

Eclipse Vert.x Open source project started in 2012 Created by Tim Fox Eclipse / Apache licensing A toolkit for building reactive applications for the JVM ! https://vertx.io " vertx_project

Slide 4

Slide 4 text

Installation Java 8 Vert.x is a set of jars on Maven Central Unopinionated : your build, your IDE CLI vertx tool

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

Outline Vert.x core 101 Networking with Vert.x Reactive programming with RxJava Boiler Vroom Ecosystem (we may skip if time runs out)

Slide 8

Slide 8 text

Live coding: simple server and test

Slide 9

Slide 9 text

Vert.x core — 101

Slide 10

Slide 10 text

Vert.x Concurrency Model

Slide 11

Slide 11 text

while (isRunning) { String line = bufferedReader.readLine(); switch (line.substring(0, 4)) { case "ECHO": bufferedWriter.write(line); break // ... // other cases ( ...) // ... default: bufferedWriter.write("UNKW Unknown command"); } }

Slide 12

Slide 12 text

x 1000 = %

Slide 13

Slide 13 text

String line = bufferedReader.readLine(); switch (line.substring(0, 4)) { case "ECHO": bufferedWriter.write(line); break // (…) & C1 C2

Slide 14

Slide 14 text

C1 “When you have a line of text, call C2” Something else with no blocking call either C2

Slide 15

Slide 15 text

Events Thread Event Loop

Slide 16

Slide 16 text

Reactor Multi-reactor

Slide 17

Slide 17 text

 2 event-loops per CPU core by default

Slide 18

Slide 18 text

 Verticles ( ( ( public class SomeVerticle extends AbstractVerticle { @Override public void start() throws Exception { } @Override public void stop() throws Exception { } } class SomeVerticle : AbstractVerticle() { override fun start() { } override fun stop() { } } exports.vertxStart = function() { } exports.vertxStop = function() { }

Slide 19

Slide 19 text

( ) Configuration ( ( Network events come from acceptor threads Verticle Deploy Deploy

Slide 20

Slide 20 text

( Worker thread pool Blocking task * executeBlocking Result

Slide 21

Slide 21 text

( “Regular verticle” (on event-loop) * ( Worker verticle (1 thread) Multi-thread worker verticle ( Worker pool

Slide 22

Slide 22 text

Message passing on the event bus

Slide 23

Slide 23 text

( ( Http server verticle Database client verticle  ?

Slide 24

Slide 24 text

( - ( Http server verticle Database client verticle  Event Bus . “Details for user 1234?” Send to “user.db”

Slide 25

Slide 25 text

( - ( Http server verticle Database client verticle  Event Bus . “Details for user 1234?” Send to “user.db” Consume from “user.db”

Slide 26

Slide 26 text

( - ( Http server verticle Database client verticle  Event Bus . - “Details for user 1234?” “{data}”

Slide 27

Slide 27 text

. Asynchronous messaging “foo.bar”, “foo-bar”, “foo/bar”, … Point to point (with possible response back) Publish / subscribe

Slide 28

Slide 28 text

. Distributed across Vert.x nodes Hazelcast, Ignite, Inﬁnispan, … TCP bridge interface Go, Python, C, JavaScript, Swift, C#, … SockJS bridge Seamless frontend / backend messaging

Slide 29

Slide 29 text

- Headers DeliveryOptions (e.g., timeouts) Body Address Reply address

Slide 30

Slide 30 text

- “Primitive” types String, int, double, … JSON Object/Array Polyglot applications, clean boundaries Custom codecs For advanced usages

Slide 31

Slide 31 text

Live coding: event bus demo

Slide 32

Slide 32 text

switch (message.headers().get("action")) { case "all-pages": fetchAllPages(message); break; case "get-page": fetchPage(message); break; case "create-page": createPage(message); break; case "save-page": savePage(message); break; case "delete-page": deletePage(message); break; default: message.fail(ErrorCodes.BAD_ACTION.ordinal(), "Bad action: " + action); }

Slide 33

Slide 33 text

private void deletePage(Message message) { dbClient.getConnection(car -> { if (car.succeeded()) { SQLConnection connection = car.result(); JsonArray data = new JsonArray().add(message.body().getString("id")); connection.updateWithParams(sqlQueries.get(SqlQuery.DELETE_PAGE), data, res -> { connection.close(); if (res.succeeded()) { message.reply(new JsonObject().put("result", "ok")); } else { reportQueryError(message, res.cause()); } }); } else { reportQueryError(message, car.cause()); } }); }

Slide 34

Slide 34 text

Slide 35

Slide 35 text

@ProxyGen public interface WikiDatabaseService { // ( ...) @Fluent WikiDatabaseService savePage(int id, String markdown, Handler> resultHandler); @Fluent WikiDatabaseService deletePage(int id, Handler> resultHandler); static WikiDatabaseService createProxy(Vertx vertx, String address) { return new WikiDatabaseServiceVertxEBProxy(vertx, address); } // ( ...) } Proxy + handler source code will be generated Parameters from a JSON document Handlers for replies Generated proxy

Slide 36

Slide 36 text

dbService = WikiDatabaseService.createProxy(vertx, "wikidb.queue"); private void pageDeletionHandler(RoutingContext context) { dbService.deletePage(Integer.valueOf(context.request().getParam("id")), reply -> { if (reply.succeeded()) { context.response().setStatusCode(303); context.response().putHeader("Location", "/"); context.response().end(); } else { context.fail(reply.cause()); } }); }

Slide 37

Slide 37 text

Networking with Vert.x

Slide 38

Slide 38 text

Networking with Vert.x Vert.x Core TCP, HTTP/1, HTTP/2, WebSocket, UDP, DNS Vert.x Web SockJS MQTT Server

Slide 39

Slide 39 text

Networking with Vert.x SSL/TLS Native SSL Asynchronous DNS resolver Proxy socks4, socks5, HTTP connect Metrics

Slide 40

Slide 40 text

HTTP with Vert.x Server and client! HTTP/1, HTTP/2 Websockets SockJS Persistent connections

Slide 41

Slide 41 text

Server concurrency Mongo EventLoop

Slide 42

Slide 42 text

0 0 0

Slide 43

Slide 43 text

request.handler(data -> {  s3request.write(data);  }); 

Slide 44

Slide 44 text

request.handler(data -> {  s3request.write(data);  }); 

Slide 45

Slide 45 text

request.handler(data -> {  s3request.write(data);  }); 

Slide 46

Slide 46 text

Back-pressure signal Back-pressure propagates as a signal between systems in protocols network Inter Process communication pipes threads etc…

Slide 47

Slide 47 text

Back-pressure Mechanism in protocols to slow down the data ﬂow between a producer and a consumer When demand > capacity : queue or drop packets

Slide 48

Slide 48 text

Back-pressure in protocols TCP HTTP/2 Websockets / SockJS AMQP OS pipes etc…

Slide 49

Slide 49 text

8kb // Might block when the buffer is empty  int n = request.getInputStream().read(data);    // Got some bytes // Might block when the buffer is full  s3request.getOutputStream().write(data, 0, n); // Queued for sending 8kb

Slide 50

Slide 50 text

request.handler(data -> {  if (!s3request.writeQueueFull()) { s3request.write(data); } });  full

Slide 51

Slide 51 text

request.handler(data -> {  s3request.write(data); if (s3request.writeQueueFull()) { // stop reading from the request request.pause(); } });  full pause

Slide 52

Slide 52 text

request.handler(data -> {  s3request.write(data); if (s3request.writeQueueFull()) { request.pause(); s3request.drainHandler(v -> { socket.resume(); }); } });  drain resume

Slide 53

Slide 53 text

Read stream public interface ReadStream {    void pause();    void resume();    void handler(Handler handler);    void endHandler(Handler handler);    }

Slide 54

Slide 54 text

Write stream public interface WriteStream {    void write(T data);    boolean writeQueueFull();    void drainHandler(Handler handler);    void end();    }

Slide 55

Slide 55 text

Pumping Pump pump = Pump.pump(request, s3request); pump.start();  request.endHandler(v -> { pump.stop(); s3Request.end(); }); 

Slide 56

Slide 56 text

Uniﬁed back-pressure TCP streams HTTP client/server request/responses WebSocket Async ﬁle Kafka client consumer/producer SockJS OS pipes (…)

Slide 57

Slide 57 text

Reactive Programming with Vert.x and RxJava

Slide 58

Slide 58 text

RxJava Data and events ﬂows Great at organizing transformation of data and coordination of events It makes most sense when many sources of events are involved

Slide 59

Slide 59 text

Motivation Future> is not appropriate Dealing with latencies Functional programming inﬂuence

Slide 60

Slide 60 text

Push based subscribe ➊ ➋push Observer Observable

Slide 61

Slide 61 text

Iterable / Observable try {  for (String item : it) { ➊  } ➌  } catch (Throwable e) { ➋  } observable.subscribe(item -> {  ➊ // onNext  }, error -> {  ➋ // onError  }, () -> {  ➌ // onCompleted  });

Slide 62

Slide 62 text

0 1 0..n Reactive Completable Single Observable Interactive void T Iterable

Slide 63

Slide 63 text

Reactive pull back pressure subscribe ➊ ➌push Observer Observable ➋request

Slide 64

Slide 64 text

Operators

Slide 65

Slide 65 text

Rxiﬁed APIs Each API type (annotated with @VertxGen) has its preﬁx io.vertx replaced by io.vertx.rxjava io.vertx.core.Vertx 1 io.vertx.rxjava.Vertx etc…

Slide 66

Slide 66 text

Rxiﬁed HttpServerRequest server.requestHandler(request -> {    Observable obs = request.toObservable();    obs.subscribe(buffer -> {  // A new buffer  }, err -> {  // Something wrong happened  }, () -> {  // Done  });  });

Slide 67

Slide 67 text

Rxiﬁed Handler void listen(int port, Handler> ar)    Single rxListen(int port);

Slide 68

Slide 68 text

Live coding: API gateway with RxJava

Slide 69

Slide 69 text

RxJava 2 Better performances Based on reactive-streams Null values forbidden More reactive types Observable / Flowable Single / Maybe / Completable Prototype at https://github.com/vert-x3/vertx-rx

Slide 70

Slide 70 text

Boiler Vroom 2

Slide 71

Slide 71 text

Boiler Vroom, unplugged 3

Slide 72

Slide 72 text

MIDI Signals Channel control Note on / off Start / Stop Pitchbend Clock (…) https://github.com/cotejp/webmidi https://webaudio.github.io/web-midi-api/ https://github.com/jponge/webmidi-sequencer # https://github.com/jponge/boiler-vroom 24 clock messages per bar

Slide 73

Slide 73 text

MIDI In / Out mappings

Slide 74

Slide 74 text

4 . WebMidi JSON / Event Bus

Slide 75

Slide 75 text

5 6 7 8 IceCast VLC 4 Chrome / DJ Booth app Clients 9 NuProcess MIDI Event Bus Event Bus

Slide 76

Slide 76 text

5 6 7 8 IceCast VLC Clients 9 OGG/Vorbis OGG/Vorbis MP3 MP3 HTTP 1.0 HTTP 1.1 HTTP Client Event bus → Chunked HTTP

Slide 77

Slide 77 text

No content

Slide 78

Slide 78 text

(Code walkthrough)

Slide 79

Slide 79 text

(Reactive) Ecosystem

Slide 80

Slide 80 text

Databases • MySQL, PostgreSQL • JDBC • MongoDB • Redis

Slide 81

Slide 81 text

Authentication / Security • Auth: • Apache Shiro (LDAP, properties, …) • JDBC • MongoDB • OAuth (+ providers) • Json Web Tokens (JWT)

Slide 82

Slide 82 text

Metrics • DropWizard • Hawkular • Health Checks

Slide 83

Slide 83 text

Messaging / Integration • AMQP 1.0 • STOMP • SMTP • Kafka • RabbitMQ • Camel • JCA

Slide 84

Slide 84 text

Microservices • Discovery • Circuit breaker • Conﬁg • Consul • Kubernetes • GRPC

Slide 85

Slide 85 text

Clustering • Hazelcast • Apache Ignite • Inﬁnispan • Apache Zookeeper

Slide 86

Slide 86 text

Outro

Slide 87

Slide 87 text

Vert.x Awesome Vert.x Stack Vert.x Core

Slide 88

Slide 88 text

Vert.x stack

Slide 89

Slide 89 text

https: //youtu.be/ZkWsilpiSqw : Applications réactives avec Eclipse Vert.x ; Building Reactive Microservices in Java https: //goo.gl/ep6yB9 https: //youtu.be/ApGNp4uHKdY : Microservices réactifs avec Eclipse Vert.x et Kubernetes ; Preview of a guide for Java developers https: //goo.gl/yJ7OTb

Slide 90

Slide 90 text

@jponge " https: //github.com/jponge/vertx-poitoujug-june-2017-code #