Slide 1

Slide 1 text

Streaming Data Workshop

Slide 2

Slide 2 text

Who are we ? Thomas @tsegismont Galder @galderz Katia @karesti

Slide 3

Slide 3 text

Agenda ● Install VM ● Overview on Streaming Data Architecture ● Micro introduction to ○ Eclipse Vert.x ○ Infinispan ○ Openshift ● Warmup ● Workshop ● Wrap-up

Slide 4

Slide 4 text

VM installation - Mouse TOUCHPAD

Slide 5

Slide 5 text

VM installation - Cores

Slide 6

Slide 6 text

VM installation - Enable 3D

Slide 7

Slide 7 text

VM installation - Guest Additions - Add Optical Drive

Slide 8

Slide 8 text

Optional : Copy / Paste both ways

Slide 9

Slide 9 text

Start the VM

Slide 10

Slide 10 text

Virtual Box / Devices / Insert Guest Additions CD...

Slide 11

Slide 11 text

Update workshop in VM cd streaming-data-workshop git fetch origin git pull

Slide 12

Slide 12 text

Hosts - config for 100% offline /etc/hosts 127.0.0.1 datagrid-visualizer-myproject.127.0.0.1.nip.io 127.0.0.1 simple-web-app-myproject.127.0.0.1.nip.io 127.0.0.1 delayed-listener-myproject.127.0.0.1.nip.io 127.0.0.1 delayed-trains-myproject.127.0.0.1.nip.io 127.0.0.1 workshop-main-myproject.127.0.0.1.nip.io

Slide 13

Slide 13 text

In many scenarios the computation part of the system is operating in a non-hard real-time fashion, however, the clients may not be consuming the data in real-time, due to network delays, application design, or perhaps a client application is not even running. Put another way, what we really have is a non-hard real-time service with clients that consume data when they need it. This is a streaming data system, a non-hard real-time system that makes its data available at the moment a client application needs it, it is not soft or near, it is streaming. Streaming Data - Andrew G. Psaltis - Manning Publications. Streaming Data System - Definition

Slide 14

Slide 14 text

Streaming Data Architecture - Layers Collection Tier Message Queueing Tier Analysis Tier In-Memory Data Store Data Access Tier Long Term Storage Browser, Device, Machine ... Browser, Device, Machine ...

Slide 15

Slide 15 text

Streaming Data Architecture - Layers Collection Tier Message Queueing Tier Analysis Tier In-Memory Data Store Data Access Tier Long Term Storage Browser, Device, Machine ... Browser, Device, Machine ...

Slide 16

Slide 16 text

Streaming Data Architecture - Layers Collection Tier Message Queueing Tier Analysis Tier In-Memory Data Store Data Access Tier Browser, Device, Machine ... Browser, Device, Machine ... *

Slide 17

Slide 17 text

● Open-source project to create reactive applications and micro-services ● Inspired from NodeJS ○ Reactor pattern ○ events ● Unopinionated ● Non blocking ● Reactive ● Scalable Eclipse Vert.x

Slide 18

Slide 18 text

● Open-source project mainly maintained by Red Hat ● 9.1 Last Stable Release, version 9.2 in development ● Cache / In-Memory Datagrid ○ Local Cache ○ HIbernate Cache ○ Replicated Caches ○ Distributed Caches ○ Transactions ○ Listeners ○ Continuous Query ○ Clustered Data Structures (Multimaps, Counters, Locks … and more coming) ○ Client/Server Architecture - HotRod, Memcache, REST … ○ … and more ... Infinispan

Slide 19

Slide 19 text

● Container platform ● Kubernetes plus extras ● Build pipelines ● Auto-Scaling ● Service Catalog ● Online and Hosted Openshift

Slide 20

Slide 20 text

Warmup

Slide 21

Slide 21 text

● Vert.x web application ● Infinispan cluster ● Deploy in openshift Ex 1: Your first application

Slide 22

Slide 22 text

Demo

Slide 23

Slide 23 text

Model - 2 real streams Train Train Position Stop Timed Position Station GeoLoc Timestamp Lat/Lng trainId delay name id 1..N

Slide 24

Slide 24 text

● Microservices ● Event bus Vert.x

Slide 25

Slide 25 text

RxJava - Vert.x Rxfied API

Slide 26

Slide 26 text

● Remote caches via HotRod Protocol ● Continuous query ● Ickle query Infinispan

Slide 27

Slide 27 text

Hands On