Slide 1
Slide 1 text
Streaming Data Workshop
Codemotion Madrid 2017
Slide 2
Slide 2 text
Who are we ?
Thomas
@tsegismont
Galder
@galderz
Katia
@karesti
Slide 3
Slide 3 text
Agenda
● Install VM
● Overview on Streaming Data Architecture
● Micro introduction to
○ Eclipse Vert.x
○ Infinispan
○ Openshift
● Warmup
● Workshop
● Wrap-up
Slide 4
Slide 4 text
In many scenarios the computation part of the system is operating in a
non-hard real-time fashion, however, the clients may not be consuming the
data in real-time, due to network delays, application design, or perhaps a client
application is not even running. Put another way, what we really have is a
non-hard real-time service with clients that consume data when they need it.
This is a streaming data system, a non-hard real-time system that makes its
data available at the moment a client application needs it, it is not soft or near,
it is streaming.
Streaming Data - Andrew G. Psaltis - Manning Publications.
Streaming Data System - Definition
Slide 5
Slide 5 text
Streaming Data Architecture - Layers
Collection Tier
Message
Queueing Tier
Analysis Tier
In-Memory Data
Store
Data Access
Tier
Long Term
Storage
Browser,
Device,
Machine
...
Browser,
Device,
Machine
...
Slide 6
Slide 6 text
Streaming Data Architecture - Layers
Collection Tier
Message
Queueing Tier
Analysis Tier
In-Memory Data
Store
Data Access
Tier
Long Term
Storage
Browser,
Device,
Machine
...
Browser,
Device,
Machine
...
Slide 7
Slide 7 text
Streaming Data Architecture - Layers
Collection Tier
Message
Queueing Tier
Analysis Tier
In-Memory Data
Store
Data Access
Tier
Browser,
Device,
Machine
...
Browser,
Device,
Machine
...
*
Slide 8
Slide 8 text
● Open-source project to create reactive applications and micro-services
● Inspired from NodeJS
○ Reactor pattern
○ events
● Unopinionated
● Non blocking
● Reactive
● Scalable
Eclipse Vert.x
Slide 9
Slide 9 text
● Open-source project mainly maintained by Red Hat
● 9.1 Last Stable Release, version 9.2 in development
● Cache / In-Memory Datagrid
○ Local Cache
○ HIbernate Cache
○ Replicated Caches
○ Distributed Caches
○ Transactions
○ Listeners
○ Continuous Query
○ Clustered Data Structures (Multimaps, Counters, Locks … and more coming)
○ Client/Server Architecture - HotRod, Memcache, REST …
○ … and more ...
Infinispan
Slide 10
Slide 10 text
● Container platform
● Kubernetes plus extras
● Build pipelines
● Auto-Scaling
● Service Catalog
● Online and Hosted
Openshift
Slide 11
Slide 11 text
Warmup
Slide 12
Slide 12 text
Model - 2 real streams
Train Train Position
Stop
Timed Position
Station
GeoLoc
Timestamp Lat/Lng
trainId
delay
name
id
1..N
Slide 13
Slide 13 text
Hands On