@ifesdjeen
Stream Processing and
Functional Programming
Slide 2
Slide 2 text
http://instana.com/
Slide 3
Slide 3 text
Introduction
Slide 4
Slide 4 text
Being Ukrainian
Slide 5
Slide 5 text
Being Ukrainian
Slide 6
Slide 6 text
Compuater Scientist
Slide 7
Slide 7 text
Evaluating Ideas
Slide 8
Slide 8 text
Talking to the People
Slide 9
Slide 9 text
Data Science
Slide 10
Slide 10 text
Going sub-second
Slide 11
Slide 11 text
An ultimate
problem-solving
algorithm
Slide 12
Slide 12 text
Identify the problem
Think Real Hard
Write down the solution
Slide 13
Slide 13 text
Functional Programming 101
Slide 14
Slide 14 text
Partial Application
Currying
Closures
Slide 15
Slide 15 text
Currying
((a → b) → c) →
(a → (b → c)) =
λf. λx. λy. f (x, y)
Slide 16
Slide 16 text
Partial Application
(a → b → c) = a → (b → c)
Slide 17
Slide 17 text
Closures
BiFunction fn =
(a, b) -> a + b;
Function> curried =
a -> {
return (b) -> a + b;
};
Slide 18
Slide 18 text
Stream Processing
Slide 19
Slide 19 text
Producer
Consumer
Pipe
Slide 20
Slide 20 text
Producer
Yields the values
Allows Consumers to subscribe
Slide 21
Slide 21 text
Consumer
Awaits for the values
Subscribes to Producer
Slide 22
Slide 22 text
Pipe
Connects consumers and producers
Slide 23
Slide 23 text
Pipe
data Step a i =
Yield a i
| Skip i
data Pipe a = Pipe (i -> Step a i) i
- - Monads in Stream type omitted for brevity
Slide 24
Slide 24 text
Stream
Combination of all
“Container”
Burrito
Slide 25
Slide 25 text
Stream is a Functor
fmap id = id
fmap (g . h) = (fmap g) . (fmap h)
class Functor f where
fmap :: (a -> b) -> f a -> f b
Functor Laws
morphism, mapping between categories
Slide 26
Slide 26 text
Stream is an Applicative
pure id <*> v = v
pure f <*> pure x = pure (f x)
u <*> pure y = pure ($ y) <*> u
u <*> (v <*> w) = pure (.) <*> u <*> v <*> w
class Functor f => Applicative f where
pure :: a -> f a
(<*>) :: f (a -> b) -> f a -> f b
Applicative Laws
Slide 27
Slide 27 text
Basic Stream Operations
map (Stream s) :: (a -> b) -> s a -> s b
filter (Stream s) :: (a -> Boolean) -> s a -> s a
slide (Stream s) :: ([a] -> [a]) -> s a -> s [a]
partition (Stream s) :: ([a] -> Boolean) -> s a -> s [a]
consume (Stream a) :: (a -> IO ()) -> s a -> s a
Slide 28
Slide 28 text
Building Blocks
Slide 29
Slide 29 text
Firehose on(final K key, Consumer consumer)
Firehose notify(final K key, final V ev)
Implementation Idea
Slide 30
Slide 30 text
Firehose firehose = new Firehose<>();
firehose.on(Key.wrap("myKey"), (v) -> { /* ... */} );
firehose.notify(Key.wrap("key1"), 1);
Example
Slide 31
Slide 31 text
Logical Consequence
Named Pipe
Slide 32
Slide 32 text
SRC source, DST destination, Function mapper
Named Pipe
Slide 33
Slide 33 text
key1 key2 key3
Intuition
Slide 34
Slide 34 text
(SRC source, DST destination, Function mapper) -> {
firehose.on(source, (SRC key, V value) -> {
firehose.notify(destination, mapper.apply(value));
}
});
return new NamedPipe<>(firehose);
}
Implementation Idea
Slide 35
Slide 35 text
NamedPipe intPipe = new NamedPipe<>();
intPipe.map("key1",
"key2", (i) -> i + 1);
intPipe.map("key2",
"key3", (i) -> i * 2);
intPipe.consume("key2",
System.out::println);
Example
Slide 36
Slide 36 text
Reducing Boilerplate
Anonymous Pipe
Slide 37
Slide 37 text
Function mapper
SRC and DST are implicit
Anonymous Pipe
Slide 38
Slide 38 text
key key’ key’’
Intuition
Slide 39
Slide 39 text
private final Key upstream;
private final NamedPipe pipe;
AnonymousPipe map(Function mapper) {
Key downstream = upstream.derive();
pipe.map(upstream, downstream, mapper);
return new AnonymousPipe<>(downstream, pipe);
}
Implementation Idea
Slide 40
Slide 40 text
Example
NamedPipe pipe = new NamedPipe<>();
pipe.anonymous("key")
.map((i) -> i + 1)
.map(i -> i * 2)
.consume(System.out::println);
pipe.notify("key", 1);