Eclipse Vert.x for Dj fun and for profit!

Slide 1

Slide 1 text

Eclipse Vert.x for Dj fun and for profit! Speaker : Julien Ponge - @jponge

Slide 2

Slide 2 text

Julien Ponge Maitre de Conférences “Delegated consultant to Red Hat” on Vert.x Eclipse Golo + extensive F/OSS background ! https://julien.ponge.org/ " @jponge # @jponge  https://www.mixcloud.com/hclcast/

Slide 3

Slide 3 text

Outline ✓ Reactive? Vert.x? ✓ Vert.x core ✓ Reactive programming with Vert.x / RxJava ✓ BoilerVroom

Slide 4

Slide 4 text

Reactive? Vert.x?

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

Application

Slide 7

Slide 7 text

Software Messages Requests Metrics Availability

Slide 8

Slide 8 text

Reactive systems Reactive streams Reactive programming Reactive “Responding to stimuli” Manifesto, Actor, Messages Resilience, Elasticity, Scalability, Asynchronous, non-blocking Data flow Back-pressure Non-blocking Data flow Events, Observable Spreadsheets Akka, Vert.x Akka Streams, Rx v2, Reactor, Vert.x Reactor, Reactive Spring, RxJava, Vert.x

Slide 9

Slide 9 text

Eclipse Vert.x Open source project started in 2012 Created by Tim Fox Eclipse / Apache licensing A toolkit for building reactive applications for the JVM Built on top of Netty ! https://vertx.io " @vertx_project

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

(demo) “Hello world in action”

Slide 12

Slide 12 text

Vert.x core — 101

Slide 13

Slide 13 text

Outline ✓ Vert.x concurrency model ✓ Message passing on the event bus ✓ Dealing with asynchronous events ✓ Failover demo

Slide 14

Slide 14 text

Vert.x Concurrency Model

Slide 15

Slide 15 text

while (isRunning) { String line = bufferedReader.readLine(); switch (line.substring(0, 4)) { case "ECHO": bufferedWriter.write(line); break // ... // other cases ( ...) // ... default: bufferedWriter.write("UNKW Unknown command"); } }

Slide 16

Slide 16 text

x 1000 = %

Slide 17

Slide 17 text

C1 “When you have a line of text, call C2” Something else with no blocking call either C2

Slide 18

Slide 18 text

Events Thread Event Loop

Slide 19

Slide 19 text

 2 event-loops per CPU core by default

Slide 20

Slide 20 text

 Verticles ' ' ' public class SomeVerticle extends AbstractVerticle { @Override public void start() throws Exception { } @Override public void stop() throws Exception { } } class SomeVerticle : AbstractVerticle() { override fun start() { } override fun stop() { } } exports.vertxStart = function() { } exports.vertxStop = function() { }

Slide 21

Slide 21 text

' ( Configuration ' ' Network events come from acceptor threads Verticle Deploy Deploy

Slide 22

Slide 22 text

' Worker thread pool Blocking task ) executeBlocking Result

Slide 23

Slide 23 text

' “Regular verticle” (on event-loop) ) ' Worker verticle (1 thread) Multi-thread worker verticle ' Worker pool

Slide 24

Slide 24 text

Message passing on the event bus

Slide 25

Slide 25 text

' ' Http server verticle Database client verticle  ?

Slide 26

Slide 26 text

' , ' Http server verticle Database client verticle  Event Bus - “Details for user 1234?” Send to “user.db”

Slide 27

Slide 27 text

' , ' Http server verticle Database client verticle  Event Bus - “Details for user 1234?” Send to “user.db” Consume from “user.db”

Slide 28

Slide 28 text

' , ' Http server verticle Database client verticle  Event Bus - , “Details for user 1234?” “{data}”

Slide 29

Slide 29 text

- Asynchronous messaging “foo.bar”, “foo-bar”, “foo/bar”, … Point to point (with possible response back) Publish / subscribe

Slide 30

Slide 30 text

- Distributed across Vert.x nodes Hazelcast, Ignite, Inﬁnispan, … TCP bridge interface Go, Python, C, JavaScript, Swift, C#, … SockJS bridge Seamless frontend / backend messaging

Slide 31

Slide 31 text

, Headers DeliveryOptions (e.g., timeouts) Body Address Reply address

Slide 32

Slide 32 text

, “Primitive” types String, int, double, … JSON Object/Array Polyglot applications, clean boundaries Custom codecs For advanced usages

Slide 33

Slide 33 text

EventBus eb = vertx.eventBus(); eb.consumer("ping-address", message -> { System.out.println("Received message: " + message.body()); message.reply("pong!"); }); EventBus eb = vertx.eventBus(); vertx.setPeriodic(1000, v -> { eb.send("ping-address", "ping!", reply -> { if (reply.succeeded()) { System.out.println("Received reply " + reply.result().body()); } else { System.out.println("No reply"); } }); });

Slide 34

Slide 34 text

switch (message.headers().get("action")) { case "all-pages": fetchAllPages(message); break; case "get-page": fetchPage(message); break; case "create-page": createPage(message); break; case "save-page": savePage(message); break; case "delete-page": deletePage(message); break; default: message.fail(ErrorCodes.BAD_ACTION.ordinal(), "Bad action: " + action); }

Slide 35

Slide 35 text

private void deletePage(Message message) { dbClient.getConnection(car -> { if (car.succeeded()) { SQLConnection connection = car.result(); JsonArray data = new JsonArray().add(message.body().getString("id")); connection.updateWithParams(sqlQueries.get(SqlQuery.DELETE_PAGE), data, res -> { connection.close(); if (res.succeeded()) { message.reply(new JsonObject().put("result", "ok")); } else { reportQueryError(message, res.cause()); } }); } else { reportQueryError(message, car.cause()); } }); }

Slide 36

Slide 36 text

Slide 37

Slide 37 text

@ProxyGen public interface WikiDatabaseService { // ( ...) @Fluent WikiDatabaseService savePage(int id, String markdown, Handler> resultHandler); @Fluent WikiDatabaseService deletePage(int id, Handler> resultHandler); static WikiDatabaseService createProxy(Vertx vertx, String address) { return new WikiDatabaseServiceVertxEBProxy(vertx, address); } // ( ...) } Proxy + handler source code will be generated Parameters from a JSON document Handlers for replies Generated proxy

Slide 38

Slide 38 text

dbService = WikiDatabaseService.createProxy(vertx, "wikidb.queue"); private void pageDeletionHandler(RoutingContext context) { dbService.deletePage(Integer.valueOf(context.request().getParam("id")), reply -> { if (reply.succeeded()) { context.response().setStatusCode(303); context.response().putHeader("Location", "/"); context.response().end(); } else { context.fail(reply.cause()); } }); }

Slide 39

Slide 39 text

Dealing with asynchronous events

Slide 40

Slide 40 text

Callbacks RxJava 1 + 2 Quasar (vertx-sync) Kotlin coroutines (core) (codegen) (codegen) (codegen) (codegen)

Slide 41

Slide 41 text

foo.a(1, res1 -> { if (res1.succeeded()) { bar.b("abc", 1, res2 -> { if (res.succeeded()) { baz.c(res3 -> { dosomething(res1, res2, res3, res4 -> { // (...) }); }); } }); } }); “Callback hell”

Slide 42

Slide 42 text

jdbc.rxGetConnection().flatMap(conn -> { // Now chain some statements using flatmap composition Single resa = conn.rxUpdate("CREATE TABLE test(col VARCHAR(20))") .flatMap(result -> conn.rxUpdate("INSERT INTO test (col) VALUES ('val1')")) .flatMap(result -> conn.rxUpdate("INSERT INTO test (col) VALUES ('val2')")) .flatMap(result -> conn.rxQuery("SELECT * FROM test")); return resa.doAfterTerminate(conn ::close); }).subscribe(resultSet -> { // Subscribe to the final result System.out.println("Results : " + resultSet.getRows()); }, err -> { System.out.println("Database problem"); err.printStackTrace(); });

Slide 43

Slide 43 text

try (SQLConnection conn = awaitResult(jdbc ::getConnection)) { // Create a table Void v = awaitResult(h -> conn.execute("CREATE TABLE test(col VARCHAR(20))", h)); // Insert some stuff for (int i = 0; i < 10; i ++) { int ii = i; UpdateResult res = awaitResult(h -> conn.update("INSERT INTO test (col) VALUES ('val" + ii + "')", h)); System.out.println("Rows updated: " + res.getUpdated()); } // Select the results ResultSet res = awaitResult(h -> conn.query("SELECT * FROM test", h)); System.out.println("Selected " + res.getNumRows() + " results"); res.getResults().forEach(System.out ::println); }

Slide 44

Slide 44 text

val consumer = vertx.eventBus().localConsumer("a.b.c") consumer.handler { message -> println("Consumer received: ${message.body()}") message.reply("pong") } // Wait until the consumer has properly registered asyncResult { h -> consumer.completionHandler(h) } // Send a message and wait for a reply val reply = asyncResult> { h -> vertx.eventBus().send("a.b.c", "ping", h) } println("Reply received: ${reply.body()}")

Slide 45

Slide 45 text

Reactive Programming with Vert.x and RxJava

Slide 46

Slide 46 text

RxJava Data and events ﬂows Organising transformation of data and coordination of events Makes most sense with many sources of events

Slide 47

Slide 47 text

Motivation Future> is not appropriate Dealing with latencies Functional programming inﬂuence

Slide 48

Slide 48 text

Push based subscribe ➊ ➋push Observer Observable

Slide 49

Slide 49 text

Iterable / Observable try {  for (String item : it) { ➊  } ➌  } catch (Throwable e) { ➋  } observable.subscribe(item -> {  ➊ // onNext  }, error -> {  ➋ // onError  }, () -> {  ➌ // onCompleted  });

Slide 50

Slide 50 text

0 1 0..n Reactive Completable Single Observable Interactive void T Iterable

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

No content

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

Rxiﬁed APIs Each API type (annotated with @VertxGen) has its preﬁx io.vertx replaced by io.vertx.rxjava io.vertx.core.Vertx / io.vertx.rxjava.Vertx etc…

Slide 56

Slide 56 text

Rxiﬁed Handler void listen(int port, Handler> ar)    Single rxListen(int port);

Slide 57

Slide 57 text

(demo) RxJava

Slide 58

Slide 58 text

RxJava 2 Better performances Based on reactive-streams Null values forbidden More reactive types Observable / Flowable Single / Maybe / Completable Upcoming in 3.5.0

Slide 59

Slide 59 text

Boiler Vroom 0

Slide 60

Slide 60 text

Boiler Vroom, unplugged 1

Slide 61

Slide 61 text

MIDI Signals Channel control Note on / off Start / Stop Pitchbend Clock (…) https://github.com/cotejp/webmidi https://webaudio.github.io/web-midi-api/ https://github.com/jponge/webmidi-sequencer # https://github.com/jponge/boiler-vroom 24 clock messages per bar

Slide 62

Slide 62 text

MIDI In / Out mappings

Slide 63

Slide 63 text

2 - WebMidi JSON / Event Bus

Slide 64

Slide 64 text

3 4 5 6 IceCast VLC 2 Chrome / DJ Booth app Clients 7 NuProcess MIDI Event Bus Event Bus

Slide 65

Slide 65 text

3 4 5 6 IceCast VLC Clients 7 OGG/Vorbis OGG/Vorbis MP3 MP3 HTTP 1.0 HTTP 1.1 HTTP Client Event bus → Chunked HTTP

Slide 66

Slide 66 text

No content

Slide 67

Slide 67 text

Future icecastFuture = Future.future(); icecastProcess = Process.create(vertx, "icecast", asList("-c", “etc/icecast.xml")); icecastProcess.start(process -> icecastFuture.complete()); icecastProcess.exitHandler(exitCode -> { String message = "icecast exited with status code " + exitCode; if (!icecastFuture.isComplete()) { icecastFuture.fail(message); } logger.warn(message); }); icecastProcess.stdout().handler(logger ::info); icecastProcess.stderr().handler(logger ::warn); Reactive processes

Slide 68

Slide 68 text

HttpClientOptions options = new HttpClientOptions() .setDefaultHost("localhost") .setDefaultPort(8001); vertx.createHttpClient(options) .getNow("/stream.mp3", response -> { eventBus.publish(ANNOUNCE_DESTINATION, CONNECTED_MESSAGE); logger.info("Connected to the VLC transcoder"); response.handler(buffer -> eventBus.publish("boilervroom.audiostream", buffer)); response.exceptionHandler(t -> { logger.error(t); eventBus.publish(ANNOUNCE_DESTINATION, DISCONNECTED_MESSAGE); }); response.endHandler(v -> { logger.warn("Connection closed to the VLC transcoder"); eventBus.publish(ANNOUNCE_DESTINATION, DISCONNECTED_MESSAGE); }); }); Icecast to subscribers

Slide 69

Slide 69 text

router.get("/audiostream").handler(context -> { logger.info("New streaming client: " + context.request().remoteAddress()); HttpServerResponse response = context.response(); response.setStatusCode(200); response.setChunked(true); response.putHeader("Content-Type", "audio/mpeg"); MessageConsumer consumer = eventBus.consumer("boilervroom.audiostream"); consumer.bodyStream().handler(buffer -> { if (!response.writeQueueFull()) { response.write(buffer); } }); ( ...) Back-pressure

Slide 70

Slide 70 text

SockJSHandler sockJSHandler = SockJSHandler.create(vertx); PermittedOptions permittedOptions = new PermittedOptions() .setAddressRegex(“boilervroom \\ ..+"); BridgeOptions bridgeOptions = new BridgeOptions() .addInboundPermitted(permittedOptions) .addOutboundPermitted(permittedOptions); sockJSHandler.bridge(bridgeOptions); router.route("/eventbus /*").handler(sockJSHandler); SockJS event bus bridge: “killer feature” SockJS Event Bus

Slide 71

Slide 71 text

const eventBus = new EventBus("/eventbus") traktorIn.addListener("controlchange", 5, (event) => { eventBus.publish("boilervroom.vu-meter", { type: "volume-level", value: event.value }) }) eventBus.publish("boilervroom.fromtraktor", { type: "filter", number: 1, value: (event.value !== 0) })

Slide 72

Slide 72 text

ITopic topic = Hazelcast.newHazelcastInstance().getTopic("vu-meter"); vertx.eventBus().consumer("boilervroom.vu-meter", event -> { JsonObject body = (JsonObject) event.body(); topic.publish(body.getInteger("value")); }); Publish audio levels using Hazelcast

Slide 73

Slide 73 text

No content

Slide 74

Slide 74 text

Outro

Slide 75

Slide 75 text

Vert.x Awesome Vert.x Stack Vert.x Core

Slide 76

Slide 76 text

No content

Slide 77

Slide 77 text

Uniﬁed end-to-end reactive model + ecosystem (not just APIs…) For all kinds of distributed applications (even the small-scale ones) Flexible toolkit, not a framework (your needs, your call)

Slide 78

Slide 78 text

https: //youtu.be/ZkWsilpiSqw 8 Applications réactives avec Eclipse Vert.x 9 Building Reactive Microservices in Java https: //goo.gl/ep6yB9 https: //youtu.be/ApGNp4uHKdY 8 Microservices réactifs avec Eclipse Vert.x et Kubernetes 9 Guide to async programming with Vert.x for Java developers https: //goo.gl/AcWW3A