Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Data Streams on ElasticSearch
Search
Siddharth Kothari
November 09, 2015
Technology
0
160
Data Streams on ElasticSearch
Midwest.io '15 talk on #datastreams #realtimeAPIs #ElasticSearch
Siddharth Kothari
November 09, 2015
Tweet
Share
More Decks by Siddharth Kothari
See All by Siddharth Kothari
Data Streams with Elasticsearch
siddharthlatest
0
130
Appbase.io - Customer Deck
siddharthlatest
0
130
Appbase.io - Investor Deck
siddharthlatest
0
330
Meteor Talk: Building Mitter
siddharthlatest
1
99
Building a streaming database service
siddharthlatest
1
220
Appbase - A realtime events-based document store
siddharthlatest
1
800
Scalability Best Practices
siddharthlatest
0
180
Other Decks in Technology
See All in Technology
Navigation3でViewModelにデータを渡す方法
mikanichinose
0
220
Абьюзим random_bytes(). Фёдор Кулаков, разработчик Lamoda Tech
lamodatech
0
350
Yamla: Rustでつくるリアルタイム性を追求した機械学習基盤 / Yamla: A Rust-Based Machine Learning Platform Pursuing Real-Time Capabilities
lycorptech_jp
PRO
3
130
250627 関西Ruby会議08 前夜祭 RejectKaigi「DJ on Ruby Ver.0.1」
msykd
PRO
2
320
セキュリティの民主化は何故必要なのか_AWS WAF 運用の 10 の苦悩から学ぶ
yoh
1
180
第9回情シス転職ミートアップ_テックタッチ株式会社
forester3003
0
250
フィンテック養成勉強会#54
finengine
0
180
Observability infrastructure behind the trillion-messages scale Kafka platform
lycorptech_jp
PRO
0
140
Кто отправит outbox? Валентин Удальцов, автор канала Пых
lamodatech
0
350
Claude Code Actionを使ったコード品質改善の取り組み
potix2
PRO
6
2.4k
Liquid Glass革新とSwiftUI/UIKit進化
fumiyasac0921
0
230
より良いプロダクトの開発を目指して - 情報を中心としたプロダクト開発 #phpcon #phpcon2025
bengo4com
1
3.1k
Featured
See All Featured
It's Worth the Effort
3n
185
28k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
107
19k
Site-Speed That Sticks
csswizardry
10
660
Typedesign – Prime Four
hannesfritz
42
2.7k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.7k
The Power of CSS Pseudo Elements
geoffreycrofte
77
5.8k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
2.9k
Rebuilding a faster, lazier Slack
samanthasiow
82
9.1k
Docker and Python
trallard
44
3.4k
Rails Girls Zürich Keynote
gr2m
94
14k
Transcript
Data Streams on ElasticSearch - @siddharthlatest
None
We have a problem… Databases have been around for over
half a century. Indexes, Caches, Distributed, Materialized Views, Stored Procedures, Aggregations
We have a problem… Databases have been around for over
half a century. Indexes, Caches, Distributed, Materialized Views, Stored Procedures, Aggregations Not Unix Philosophy!
NoSQL: Webscale Movement - 2000s Static Webpages - 1990
NoSQL: Webscale Movement - 2000s Static Webpages - 1990 #RealtimeWeb
- Today
#blackfriday #SxSW2015 #worldcup #realtimeweb
None
Topics: 1. What is the point of “data streams”? 2.
ElasticSearch as the Query Layer 3. Data Streams in Action 4. Seriously, where would I use this 5. Scale 6. The Next 5 Years
None
None
None
None
Visualizing Percolation X
None
None
Topics: 1. What is the point of “data streams”? 2.
ElasticSearch as the Query Layer 3. Data Streams in Action 4. Seriously, where would I use this 5. Scale 6. The Next 5 Years
Appbase.io: Data Streams + ElasticSearch
create a continuous query interface
Tweet with hashtag -> “#midwestio” https://github.com/siddharthlatest/twitter-stream-midwestio
Tweet with hashtag -> “#midwestio” client.stream('statuses/filter', {track: 'midwestio', language: 'en'},
function(stream) { stream.on('data', function(tweet) { appbaseRef.index({ "type": "onlymidwestio", "body": tweet }); }); }); Indexes incoming streams from twitter.
Live Tweets Browser
var response = appbase.searchStream({ type: "onlymidwestio", body: {query: {match_all: {}}}
}); response.on('data', function(res) { var div = document.getElementById('tweetshero'); div.innerHTML = res._source.text + "<br><br>" + div.innerHTML; }); Live MidwestIO Tweets - http://midwest.appbase.io
appbaseio.github.io/meetupblast
Topics: 1. What is the point of “data streams”? 2.
ElasticSearch as the Query Layer 3. Data Streams in Action 4. Seriously, where would I use this 5. Scale 6. The Next 5 Years
None
Game Leaderboards, Price Surge Newsfeeds, Geoquerying Systems
Topics: 1. What is the point of “data streams”? 2.
ElasticSearch as the Query Layer 3. Data Streams in Action 4. Seriously, where would I use this 5. Scale 6. The Next 5 Years
Game Leaderboards, Price Surge
Game Leaderboards, Price Surge
Scaling websockets, http-streams with Pushpin https://github.com/fanout/pushpin Benchmarked to handle 100k
simultaneous clients, with ~500ms avg. latency.
Topics: 1. What is the point of “data streams”? 2.
ElasticSearch as the Query Layer 3. Data Streams in Action 4. Seriously, where would I use this 5. Scale 6. The Next 5 Years
Next 5 Years Native Stream support in all major languages.
Next 5 Years Native Stream support in all major languages.
Data Streams as an interface for every DB.
Next 5 Years Native Stream support in all major languages.
Data Streams as an interface for every DB. Data Streams for processing all computations, la Dataflow model.
None