Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Data Streams on ElasticSearch
Search
Siddharth Kothari
November 09, 2015
Technology
0
160
Data Streams on ElasticSearch
Midwest.io '15 talk on #datastreams #realtimeAPIs #ElasticSearch
Siddharth Kothari
November 09, 2015
Tweet
Share
More Decks by Siddharth Kothari
See All by Siddharth Kothari
Data Streams with Elasticsearch
siddharthlatest
0
130
Appbase.io - Customer Deck
siddharthlatest
0
130
Appbase.io - Investor Deck
siddharthlatest
0
330
Meteor Talk: Building Mitter
siddharthlatest
1
99
Building a streaming database service
siddharthlatest
1
220
Appbase - A realtime events-based document store
siddharthlatest
1
800
Scalability Best Practices
siddharthlatest
0
180
Other Decks in Technology
See All in Technology
Microsoft Build 2025 技術/製品動向 for Microsoft Startup Tech Community
torumakabe
2
320
LangSmith×Webhook連携で実現するプロンプトドリブンCI/CD
sergicalsix
1
140
マーケットプレイス版Oracle WebCenter Content For OCI
oracle4engineer
PRO
3
930
開発生産性を組織全体の「生産性」へ! 部門間連携の壁を越える実践的ステップ
sudo5in5k
0
140
Lambda Web Adapterについて自分なりに理解してみた
smt7174
5
130
強化されたAmazon Location Serviceによる新機能と開発者体験
dayjournal
3
230
TechLION vol.41~MySQLユーザ会のほうから来ました / techlion41_mysql
sakaik
0
200
PHP開発者のためのSOLID原則再入門 #phpcon / PHP Conference Japan 2025
shogogg
4
910
KubeCon + CloudNativeCon Japan 2025 Recap by CA
ponkio_o
PRO
0
240
WordPressから ヘッドレスCMSへ! Storyblokへの移行プロセス
nyata
0
280
変化する開発、進化する体系時代に適応するソフトウェアエンジニアの知識と考え方(JaSST'25 Kansai)
mizunori
1
260
Geminiとv0による高速プロトタイピング
shinya337
0
160
Featured
See All Featured
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
The Art of Programming - Codeland 2020
erikaheidi
54
13k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
252
21k
Reflections from 52 weeks, 52 projects
jeffersonlam
351
20k
Facilitating Awesome Meetings
lara
54
6.4k
Site-Speed That Sticks
csswizardry
10
670
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.8k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.5k
A Tale of Four Properties
chriscoyier
160
23k
Speed Design
sergeychernyshev
32
1k
Code Reviewing Like a Champion
maltzj
524
40k
The Invisible Side of Design
smashingmag
300
51k
Transcript
Data Streams on ElasticSearch - @siddharthlatest
None
We have a problem… Databases have been around for over
half a century. Indexes, Caches, Distributed, Materialized Views, Stored Procedures, Aggregations
We have a problem… Databases have been around for over
half a century. Indexes, Caches, Distributed, Materialized Views, Stored Procedures, Aggregations Not Unix Philosophy!
NoSQL: Webscale Movement - 2000s Static Webpages - 1990
NoSQL: Webscale Movement - 2000s Static Webpages - 1990 #RealtimeWeb
- Today
#blackfriday #SxSW2015 #worldcup #realtimeweb
None
Topics: 1. What is the point of “data streams”? 2.
ElasticSearch as the Query Layer 3. Data Streams in Action 4. Seriously, where would I use this 5. Scale 6. The Next 5 Years
None
None
None
None
Visualizing Percolation X
None
None
Topics: 1. What is the point of “data streams”? 2.
ElasticSearch as the Query Layer 3. Data Streams in Action 4. Seriously, where would I use this 5. Scale 6. The Next 5 Years
Appbase.io: Data Streams + ElasticSearch
create a continuous query interface
Tweet with hashtag -> “#midwestio” https://github.com/siddharthlatest/twitter-stream-midwestio
Tweet with hashtag -> “#midwestio” client.stream('statuses/filter', {track: 'midwestio', language: 'en'},
function(stream) { stream.on('data', function(tweet) { appbaseRef.index({ "type": "onlymidwestio", "body": tweet }); }); }); Indexes incoming streams from twitter.
Live Tweets Browser
var response = appbase.searchStream({ type: "onlymidwestio", body: {query: {match_all: {}}}
}); response.on('data', function(res) { var div = document.getElementById('tweetshero'); div.innerHTML = res._source.text + "<br><br>" + div.innerHTML; }); Live MidwestIO Tweets - http://midwest.appbase.io
appbaseio.github.io/meetupblast
Topics: 1. What is the point of “data streams”? 2.
ElasticSearch as the Query Layer 3. Data Streams in Action 4. Seriously, where would I use this 5. Scale 6. The Next 5 Years
None
Game Leaderboards, Price Surge Newsfeeds, Geoquerying Systems
Topics: 1. What is the point of “data streams”? 2.
ElasticSearch as the Query Layer 3. Data Streams in Action 4. Seriously, where would I use this 5. Scale 6. The Next 5 Years
Game Leaderboards, Price Surge
Game Leaderboards, Price Surge
Scaling websockets, http-streams with Pushpin https://github.com/fanout/pushpin Benchmarked to handle 100k
simultaneous clients, with ~500ms avg. latency.
Topics: 1. What is the point of “data streams”? 2.
ElasticSearch as the Query Layer 3. Data Streams in Action 4. Seriously, where would I use this 5. Scale 6. The Next 5 Years
Next 5 Years Native Stream support in all major languages.
Next 5 Years Native Stream support in all major languages.
Data Streams as an interface for every DB.
Next 5 Years Native Stream support in all major languages.
Data Streams as an interface for every DB. Data Streams for processing all computations, la Dataflow model.
None