Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Data Streams on ElasticSearch
Search
Siddharth Kothari
November 09, 2015
Technology
0
160
Data Streams on ElasticSearch
Midwest.io '15 talk on #datastreams #realtimeAPIs #ElasticSearch
Siddharth Kothari
November 09, 2015
Tweet
Share
More Decks by Siddharth Kothari
See All by Siddharth Kothari
Data Streams with Elasticsearch
siddharthlatest
0
130
Appbase.io - Customer Deck
siddharthlatest
0
140
Appbase.io - Investor Deck
siddharthlatest
0
330
Meteor Talk: Building Mitter
siddharthlatest
1
99
Building a streaming database service
siddharthlatest
1
220
Appbase - A realtime events-based document store
siddharthlatest
1
810
Scalability Best Practices
siddharthlatest
0
180
Other Decks in Technology
See All in Technology
サンドボックス技術でAI利活用を促進する
koh_naga
0
200
Snowflake Intelligenceにはこうやって立ち向かう!クラシルが考えるAI Readyなデータ基盤と活用のためのDataOps
gappy50
0
110
現場で効くClaude Code ─ 最新動向と企業導入
takaakikakei
1
220
複数サービスを支えるマルチテナント型Batch MLプラットフォーム
lycorptech_jp
PRO
0
300
ブロックテーマ時代における、テーマの CSS について考える Toro_Unit / 2025.09.13 @ Shinshu WordPress Meetup
torounit
0
100
サラリーマンの小遣いで作るtoCサービス - Cloudflare Workersでスケールする開発戦略
shinaps
2
410
EncryptedSharedPreferences が deprecated になっちゃった!どうしよう! / Oh no! EncryptedSharedPreferences has been deprecated! What should I do?
yanzm
0
220
Firestore → Spanner 移行 を成功させた段階的移行プロセス
athug
1
450
Agile PBL at New Grads Trainings
kawaguti
PRO
1
400
エラーとアクセシビリティ
schktjm
1
1.2k
La gouvernance territoriale des données grâce à la plateforme Terreze
bluehats
0
150
AI開発ツールCreateがAnythingになったよ
tendasato
0
120
Featured
See All Featured
Building a Scalable Design System with Sketch
lauravandoore
462
33k
Automating Front-end Workflow
addyosmani
1370
200k
StorybookのUI Testing Handbookを読んだ
zakiyama
31
6.1k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
285
13k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Facilitating Awesome Meetings
lara
55
6.5k
Context Engineering - Making Every Token Count
addyosmani
1
33
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.9k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.9k
We Have a Design System, Now What?
morganepeng
53
7.8k
Optimising Largest Contentful Paint
csswizardry
37
3.4k
The Cult of Friendly URLs
andyhume
79
6.6k
Transcript
Data Streams on ElasticSearch - @siddharthlatest
None
We have a problem… Databases have been around for over
half a century. Indexes, Caches, Distributed, Materialized Views, Stored Procedures, Aggregations
We have a problem… Databases have been around for over
half a century. Indexes, Caches, Distributed, Materialized Views, Stored Procedures, Aggregations Not Unix Philosophy!
NoSQL: Webscale Movement - 2000s Static Webpages - 1990
NoSQL: Webscale Movement - 2000s Static Webpages - 1990 #RealtimeWeb
- Today
#blackfriday #SxSW2015 #worldcup #realtimeweb
None
Topics: 1. What is the point of “data streams”? 2.
ElasticSearch as the Query Layer 3. Data Streams in Action 4. Seriously, where would I use this 5. Scale 6. The Next 5 Years
None
None
None
None
Visualizing Percolation X
None
None
Topics: 1. What is the point of “data streams”? 2.
ElasticSearch as the Query Layer 3. Data Streams in Action 4. Seriously, where would I use this 5. Scale 6. The Next 5 Years
Appbase.io: Data Streams + ElasticSearch
create a continuous query interface
Tweet with hashtag -> “#midwestio” https://github.com/siddharthlatest/twitter-stream-midwestio
Tweet with hashtag -> “#midwestio” client.stream('statuses/filter', {track: 'midwestio', language: 'en'},
function(stream) { stream.on('data', function(tweet) { appbaseRef.index({ "type": "onlymidwestio", "body": tweet }); }); }); Indexes incoming streams from twitter.
Live Tweets Browser
var response = appbase.searchStream({ type: "onlymidwestio", body: {query: {match_all: {}}}
}); response.on('data', function(res) { var div = document.getElementById('tweetshero'); div.innerHTML = res._source.text + "<br><br>" + div.innerHTML; }); Live MidwestIO Tweets - http://midwest.appbase.io
appbaseio.github.io/meetupblast
Topics: 1. What is the point of “data streams”? 2.
ElasticSearch as the Query Layer 3. Data Streams in Action 4. Seriously, where would I use this 5. Scale 6. The Next 5 Years
None
Game Leaderboards, Price Surge Newsfeeds, Geoquerying Systems
Topics: 1. What is the point of “data streams”? 2.
ElasticSearch as the Query Layer 3. Data Streams in Action 4. Seriously, where would I use this 5. Scale 6. The Next 5 Years
Game Leaderboards, Price Surge
Game Leaderboards, Price Surge
Scaling websockets, http-streams with Pushpin https://github.com/fanout/pushpin Benchmarked to handle 100k
simultaneous clients, with ~500ms avg. latency.
Topics: 1. What is the point of “data streams”? 2.
ElasticSearch as the Query Layer 3. Data Streams in Action 4. Seriously, where would I use this 5. Scale 6. The Next 5 Years
Next 5 Years Native Stream support in all major languages.
Next 5 Years Native Stream support in all major languages.
Data Streams as an interface for every DB.
Next 5 Years Native Stream support in all major languages.
Data Streams as an interface for every DB. Data Streams for processing all computations, la Dataflow model.
None