480

# Advanced Bonsai Workshop #1: From databases to data streams

The first part of a workshop series on the Bonsai programming language. These lecture slides include a general introduction to data stream and reactive programming using Rx, as well as a tutorial on how to create your own Bonsai nodes.

A video presentation of these slides is available at:

January 18, 2016

## Transcript

1. ### Advanced Bonsai Workshop https://bitbucket.org/horizongir/bonsai January 18-21, 2016 Bonsai = Rx

+ VPL + Packages

3. ### Workshop Programme 1. From databases to data streams 2. Bonsai

toolboxes and visual editor 3. Observable combinators: An algebra for data streams 4. Representing discrete states with data streams

Workshop
5. ### There are (almost) as many data structures as there are

algorithms Array 1D Array 2D Array 3D Linked List Tree Graph
6. ### Many bulk operations over data are independent of the details

of the data structure Ex:
7. ### Many bulk operations over data are independent of the details

of the data structure Ex:
8. ### Many bulk operations over data are independent of the details

of the data structure Ex:
9. ### Enumerable streams interface IEnumerable<T> { IEnumerator<T> GetEnumerator(); } interface IEnumerator<T>

: IDisposable { bool MoveNext(); T Current { get; } }
10. ### Programming with streams // sum of all numbers var sum

= numbers.Sum(); // sum of all numbers greater than 10 var sum = numbers.Where(x => x > 10) .Sum(); // sum of the square of all numbers greater than 10 var sum = numbers.Where(x => x > 10) .Select(x => x * x) .Sum();

14. ### Programming with streams // sum of all numbers var sum

= numbers.Sum(); // sum of all numbers greater than 10 var sum = numbers.Where(x => x > 10) .Sum(); // sum of the square of all numbers greater than 10 var sum = numbers.Where(x => x > 10) .Select(x => x * x) .Sum();
15. ### A data table is a stream of rows Name Age

Height Weight João 30 1.60 60 Maria 24 1.70 54 Pedro 27 1.90 70 Ana 44 1.55 62
16. ### Streams don’t even need a data structure class RandomStream :

IEnumerable<int> { public IEnumerator<int> GetEnumerator() { return new RandomIterator(); } class RandomIterator : IEnumerator<int> { int current; Random random = new Random(); public int Current { get { return current; } } public bool MoveNext() { current = random.Next(); return true; } public void Dispose() { } } }
17. ### Iterator blocks provide convenient syntax for creating enumerable streams IEnumerable<int>

Generator() { yield return 0; yield return 1; yield return 2; }
18. ### Iterator blocks provide convenient syntax for creating enumerable streams IEnumerable<int>

RandomNumbers() { var random = new Random(); while (true) { yield return random.Next(); } }

20. ### The real world doesn’t wait for you! Hardware Interrupts User-Interface

Events Button click Monitoring Systems
21. ### Enumerable streams interface IEnumerable<T> { IEnumerator<T> GetEnumerator(); } interface IEnumerator<T>

: IDisposable { bool MoveNext(); T Current { get; } }
22. ### Observable streams interface IObservable<T> { IDisposable Subscribe(IObserver<T> observer); } interface

IObserver<T> { void OnCompleted(); void OnNext(T value); void OnError(Exception error); }

24. ### Programming with observable streams // sum of all measurements var

sum = measurements.Sum(); // sum of all measurements greater than 10 var sum = measurements.Where(x => x > 10) .Sum(); // sum of the square of all measurements greater than 10 var sum = measurements.Where(x => x > 10) .Select(x => x * x) .Sum();

28. ### Single value sequence public class ValueSource : Source<int> { public

int Value { get; set; } public override IObservable<int> Generate() { return Observable.Return(Value); } }
29. ### Single value asynchronous sequence public class AsyncValue : Source<int> {

public int Value { get; set; } public override IObservable<int> Generate() { return Observable.Start(() => { Thread.Sleep(1000); return Value; }); } }
30. ### Multi-value sequence public class EnumerableSource : Source<int> { public override

IObservable<int> Generate() { return new[] { 0, 1, 2, 3, 4 }.ToObservable(); } }
31. ### Multi-value asynchronous sequence (generator) public class AsyncEnumerable : Source<int> {

IEnumerable<int> Generator() { for (int i = 0; i < 5; i++) { Thread.Sleep(1000); yield return i; } } public override IObservable<int> Generate() { return Generator().ToObservable(Scheduler.Default); } }
32. ### Polling a device public class CameraSource : Source<IplImage> { IEnumerable<IplImage>

FrameCapture() { using (var camera = Capture.CreateCameraCapture(0)) { while (true) { var image = camera.QueryFrame(); yield return image; } } } public override IObservable<IplImage> Generate() { return FrameCapture().ToObservable(Scheduler.Default); } }
33. ### A simple transform operator public class ScaleTransform : Transform<int, int>

{ public int Value { get; set; } public override IObservable<int> Process(IObservable<int> source) { return source.Select(x => x * Value); } }
34. ### Transform with mutable state (memory) public class AccumulatorTransform : Transform<int,

int> { public override IObservable<int> Process(IObservable<int> source) { return Observable.Defer(() => { var accumulator = 0; return source.Select(x => accumulator += x); }); } }
35. ### Encapsulating processing side-effects in a Sink public class FileSink :

Sink<int> { public string FileName { get; set; } public override IObservable<int> Process(IObservable<int> source) { return Observable.Using( () => new StreamWriter(FileName), stream => source.Do(x => stream.WriteLine(x))); } }
36. ### Workshop Project Assignment Code your own Source, Transform or Sink

for Bonsai • Web Sockets • DAQ Boards • Cameras • Joysticks • Tracking • Spike Sorting • Classifiers • File formats (NEV,HDF5) • Visualizations