Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Behavioral Databases
Search
benbjohnson
October 24, 2012
Technology
16
2.5k
Behavioral Databases
A presentation I gave at the Derailed meetup in Denver on October 24th, 2012.
benbjohnson
October 24, 2012
Tweet
Share
More Decks by benbjohnson
See All by benbjohnson
Why You Shouldn't Write a Database
benbjohnson
2
540
Static Code Analysis Using SSA
benbjohnson
6
1.7k
Writing a High Performance Database in Go
benbjohnson
11
6.7k
Raft: The Understandable Distributed Consensus Protocol
benbjohnson
106
28k
Behavioral Analytics - Understanding the "why" and "how" of your users.
benbjohnson
9
2k
New Tech - Sky
benbjohnson
2
370
Sky Analytics Pitch Deck
benbjohnson
0
580
Other Decks in Technology
See All in Technology
I could be Wrong!! - Learning from Agile Experts
kawaguti
PRO
8
3.2k
Accessibility Inspectorを活用した アプリのアクセシビリティ向上方法
hinakko
0
170
完全自律型AIエージェントとAgentic Workflow〜ワークフロー構築という現実解
pharma_x_tech
0
320
【NGK2025S】動物園(PINTO_model_zoo)に遊びに行こう
kazuhitotakahashi
0
190
🌏丸い地球を効率的に平たくする 〜🗺️地図の幾何学とWeb地図技術〜
syotasasaki593876
0
140
生成AI × 旅行 LLMを活用した旅行プラン生成・チャットボット
kominet_ava
0
150
三菱電機で社内コミュニティを立ち上げた話
kurebayashi
1
350
Git scrapingで始める継続的なデータ追跡 / Git Scraping
ohbarye
5
450
あなたの人生も変わるかも?AWS認定2つで始まったウソみたいな話
iwamot
3
820
【JAWS-UG大阪 reInvent reCap LT大会 サンバが始まったら強制終了】“1分”で初めてのソロ参戦reInventを数字で振り返りながら反省する
ttelltte
0
120
2025年に挑戦したいこと
molmolken
0
140
Oracle Base Database Service:サービス概要のご紹介
oracle4engineer
PRO
1
16k
Featured
See All Featured
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
28
2.2k
Practical Orchestrator
shlominoach
186
10k
How to Think Like a Performance Engineer
csswizardry
22
1.3k
Adopting Sorbet at Scale
ufuk
74
9.2k
Designing for humans not robots
tammielis
250
25k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
330
21k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
59k
Why You Should Never Use an ORM
jnunemaker
PRO
54
9.1k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
4 Signs Your Business is Dying
shpigford
182
22k
A Tale of Four Properties
chriscoyier
157
23k
YesSQL, Process and Tooling at Scale
rocio
170
14k
Transcript
Behavioral Databases Next Generation NoSQL Analytics By Ben Johnson
My Background
Former Oracle DBA
Former Oracle DBA Behavioral Analytics
Former Oracle DBA Data Visualization Behavioral Analytics
Why did I write a database?
Other Databases Are Too Slow Redis 100K ops/second PostgreSQL 5K
QPS/core
Processing Needed to Occur Near Data Client Server 1ms 1ms
Processing Needed to Occur Near Data Client Server 1ms 1ms
1ms 1ms
Processing Needed to Occur Near Data Client Server 1ms 1ms
1ms 1ms 1ms 1ms
Processing Needed to Occur Near Data Client Server 1ms 1ms
1ms 1ms 1ms 1ms 1ms 1ms
Processing Needed to Occur Near Data
Memory is getting cheap. Use it. 2010 $20/GB http://www.jcmit.com/memoryprice.htm
Memory is getting cheap. Use it. 2011 $10/GB 2010 $20/GB
http://www.jcmit.com/memoryprice.htm
Memory is getting cheap. Use it. 2012 $5/GB 2011 $10/GB
2010 $20/GB http://www.jcmit.com/memoryprice.htm
Traditional Databases Have a Lot Of Features I Don’t Need
Locks & Latches
Transactions
Traditional Databases are Limited to Simple Data Access Key/Value Tabular
Traditional Databases are Limited to Simple Data Access Key/Value Tabular
(Boring)
Traditional Databases Are Not Real-Time
Basics of Behavioral Data
Actions
Actions & State
Actions & State + Time
Clickstream Logs Sensor Data Financial Transactions IVR
Important Differences In Behavioral Data
Behavioral Data is Historical
Behavioral Data is Isolated
Isolated = Concurrency
Performance / Internals
Aggregates 100M events / sec / core
Optimizations Stored by object then time Memory mapped Easy, compact
data format
Simplicity Supports Int64, Double, String & Bool MessagePack Encoded C99,
No Dependencies
Writing Your Own Language (Tangent)
EQL (Event Query Language)
EQL (Event Query Language)
Qip
Qip (Doesn’t stand for anything)
What is Qip? * LLVM-backed query processing language. * JIT
compiled on the fly. * As fast as C. * Removed from build because of complexity.
GitHub Archive Visualizer (Demo)
Ruby to Sky Integration
Add Event SkyDB.add_event( new Event( object_id:1, timestamp: Time.now, action: “/sign_up”,
data: { name: “John”, age: 20 } ) )
Next Actions SkyDB.next_actions( [ “/”, “/sign_up”, “/checkout” ] )
What’s Next For Sky?
More Analytics Functions!
Cohort Analysis 1 2 3 4 5 Jan 80% 70%
65% 63% 62% Feb 83% 73% 70% 69% Mar 87% 78% 75% Apr 89% 80% May 90% Month Signed Up Months After Signing Up
DAGs Home Page Sign Up Checkout View Product Cancel Order
Awesome Open Source Analytics Tools It’s like MixPanel that you
can install!
Modules!
Predictive Behavioral Analytics What will your users do next?
Risk Analysis
Anomaly Dectection / Fraud Detection
Questions?
Contact Info @benbjohnson
[email protected]