Link
Embed
Share
Beginning
This slide
Copy link URL
Copy link URL
Copy iframe embed code
Copy iframe embed code
Copy javascript embed code
Copy javascript embed code
Share
Tweet
Share
Tweet
Slide 1
Slide 1 text
Behavioral Databases Next Generation NoSQL Analytics By Ben Johnson
Slide 2
Slide 2 text
My Background
Slide 3
Slide 3 text
Former Oracle DBA
Slide 4
Slide 4 text
Former Oracle DBA Behavioral Analytics
Slide 5
Slide 5 text
Former Oracle DBA Data Visualization Behavioral Analytics
Slide 6
Slide 6 text
Why did I write a database?
Slide 7
Slide 7 text
Other Databases Are Too Slow Redis 100K ops/second PostgreSQL 5K QPS/core
Slide 8
Slide 8 text
Processing Needed to Occur Near Data Client Server 1ms 1ms
Slide 9
Slide 9 text
Processing Needed to Occur Near Data Client Server 1ms 1ms 1ms 1ms
Slide 10
Slide 10 text
Processing Needed to Occur Near Data Client Server 1ms 1ms 1ms 1ms 1ms 1ms
Slide 11
Slide 11 text
Processing Needed to Occur Near Data Client Server 1ms 1ms 1ms 1ms 1ms 1ms 1ms 1ms
Slide 12
Slide 12 text
Processing Needed to Occur Near Data
Slide 13
Slide 13 text
Memory is getting cheap. Use it. 2010 $20/GB http://www.jcmit.com/memoryprice.htm
Slide 14
Slide 14 text
Memory is getting cheap. Use it. 2011 $10/GB 2010 $20/GB http://www.jcmit.com/memoryprice.htm
Slide 15
Slide 15 text
Memory is getting cheap. Use it. 2012 $5/GB 2011 $10/GB 2010 $20/GB http://www.jcmit.com/memoryprice.htm
Slide 16
Slide 16 text
Traditional Databases Have a Lot Of Features I Don’t Need
Slide 17
Slide 17 text
Locks & Latches
Slide 18
Slide 18 text
Transactions
Slide 19
Slide 19 text
Traditional Databases are Limited to Simple Data Access Key/Value Tabular
Slide 20
Slide 20 text
Traditional Databases are Limited to Simple Data Access Key/Value Tabular (Boring)
Slide 21
Slide 21 text
Traditional Databases Are Not Real-Time
Slide 22
Slide 22 text
Basics of Behavioral Data
Slide 23
Slide 23 text
Actions
Slide 24
Slide 24 text
Actions & State
Slide 25
Slide 25 text
Actions & State + Time
Slide 26
Slide 26 text
Clickstream Logs Sensor Data Financial Transactions IVR
Slide 27
Slide 27 text
Important Differences In Behavioral Data
Slide 28
Slide 28 text
Behavioral Data is Historical
Slide 29
Slide 29 text
Behavioral Data is Isolated
Slide 30
Slide 30 text
Isolated = Concurrency
Slide 31
Slide 31 text
Performance / Internals
Slide 32
Slide 32 text
Aggregates 100M events / sec / core
Slide 33
Slide 33 text
Optimizations Stored by object then time Memory mapped Easy, compact data format
Slide 34
Slide 34 text
Simplicity Supports Int64, Double, String & Bool MessagePack Encoded C99, No Dependencies
Slide 35
Slide 35 text
Writing Your Own Language (Tangent)
Slide 36
Slide 36 text
EQL (Event Query Language)
Slide 37
Slide 37 text
EQL (Event Query Language)
Slide 38
Slide 38 text
Qip
Slide 39
Slide 39 text
Qip (Doesn’t stand for anything)
Slide 40
Slide 40 text
What is Qip? * LLVM-backed query processing language. * JIT compiled on the fly. * As fast as C. * Removed from build because of complexity.
Slide 41
Slide 41 text
GitHub Archive Visualizer (Demo)
Slide 42
Slide 42 text
Ruby to Sky Integration
Slide 43
Slide 43 text
Add Event SkyDB.add_event( new Event( object_id:1, timestamp: Time.now, action: “/sign_up”, data: { name: “John”, age: 20 } ) )
Slide 44
Slide 44 text
Next Actions SkyDB.next_actions( [ “/”, “/sign_up”, “/checkout” ] )
Slide 45
Slide 45 text
What’s Next For Sky?
Slide 46
Slide 46 text
More Analytics Functions!
Slide 47
Slide 47 text
Cohort Analysis 1 2 3 4 5 Jan 80% 70% 65% 63% 62% Feb 83% 73% 70% 69% Mar 87% 78% 75% Apr 89% 80% May 90% Month Signed Up Months After Signing Up
Slide 48
Slide 48 text
DAGs Home Page Sign Up Checkout View Product Cancel Order
Slide 49
Slide 49 text
Awesome Open Source Analytics Tools It’s like MixPanel that you can install!
Slide 50
Slide 50 text
Modules!
Slide 51
Slide 51 text
Predictive Behavioral Analytics What will your users do next?
Slide 52
Slide 52 text
Risk Analysis
Slide 53
Slide 53 text
Anomaly Dectection / Fraud Detection
Slide 54
Slide 54 text
Questions?
Slide 55
Slide 55 text
Contact Info @benbjohnson benbjohnson@yahoo.com