Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Elasticsearch & 63 Million WordPress Sites
Search
xyu
February 06, 2014
Technology
1
200
Elasticsearch & 63 Million WordPress Sites
Overview of the Elasticsearch infrastructure that Automattic maintains to support WordPress.com.
xyu
February 06, 2014
Tweet
Share
More Decks by xyu
See All by xyu
Improving Search With One Weird Trick
xyu
0
77
Better Performance & Uptime Through Async
xyu
0
92
Dealing With Time Travelers in Analytics
xyu
0
70
Scaling WordPress Queries With Elasticsearch
xyu
1
13k
Scale Out Don't Scale Up
xyu
2
200
Understanding Language and Fixing WP Search
xyu
1
170
Jetpack Related Posts for Power Users
xyu
1
530
WP ❤️ Elasticsearch
xyu
1
350
Cloud Powered WordPress for Everyone
xyu
0
100
Other Decks in Technology
See All in Technology
Google Cloud Next '24 Recap(Cloud Run/k8s)
mokocm
0
260
家族アルバム みてねにおけるGrafana活用術 / Grafana Meetup Japan Vol.1 LT
isaoshimizu
1
840
MixIT 2024 - Pulumi : Gérer son infra avec son langage de programmation préféré
ju_hnny5
0
100
どうするコスト最適化のトレードオフ
tetsuyaooooo
1
610
FrontDoorとWebAppsを組み合わせた際のリダイレクト処理の注意点
kenichirokimura
1
610
複雑な構成要素を持つUIとの向き合い方 〜新・支出グラフでの実例〜 / B43 TECH TALK
nakamuuu
0
140
エンジニアのキャリアをちょっと楽しくする3本の軸/Three Pillars to Make an Engineer's Career More Enjoyable
kwappa
0
2.8k
いつか使うかも貯金してたらめちゃめちゃ機能が増えてた話
riyaamemiya
0
430
LangSmith入門―トレース/評価/プロンプト管理などを担うLLMアプリ開発プラットフォーム
os1ma
4
420
本当のAWS基礎
toru_kubota
0
540
Microsoft Intune 勉強会 第 2 回目
tamaiyutaro
1
220
[新卒向け研修資料] テスト文字列に「うんこ」と入れるな(2024年版)
infiniteloop_inc
4
16k
Featured
See All Featured
A Tale of Four Properties
chriscoyier
151
22k
Bootstrapping a Software Product
garrettdimon
PRO
302
110k
Teambox: Starting and Learning
jrom
128
8.4k
Docker and Python
trallard
34
2.7k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
25
2.3k
From Idea to $5000 a Month in 5 Months
shpigford
377
45k
The Mythical Team-Month
searls
216
42k
Making the Leap to Tech Lead
cromwellryan
124
8.5k
Designing on Purpose - Digital PM Summit 2013
jponch
110
6.5k
Done Done
chrislema
178
15k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
14
1.5k
Art, The Web, and Tiny UX
lynnandtonic
289
19k
Transcript
Elasticsearch & 63 Million WordPress Sites Elasticsearch Boston (Feb. 2014)
Xiao Yu Code Wrangler — Automattic @HypertextRanch
[email protected]
xyu
VaultPress Jetpack Simplenote Akismet Polldaddy Gravatar VideoPress IntenseDebate
Simperium Code Poet Cloudup
Cluster Stats • 63M Sites • 743M Documents • 12TB
Primary + Replicas • 51M Query Ops / Day • 15M Index Ops / Day 2 Major Use Cases • Global Search ! • Local Search Elasticsearch + WordPress.com
Infrastructure Layout Internal API Cache REST API PHP Node 1
Node 2 Cluster A Node 1 Node 2 Node 3 Node n Cluster B Stats
Documents & Types /index/post { blog_id: 123, post_id: 456, title:
"Search!", content: "…", blog: { lang: "en", … }, … } /index/blog { blog_id: 123, url: "www.xyu.io", follower_ids: [ 789, … ], lang: "en", indexable: true, … }
Storage Strategy • Grow Number of Indices (10M Sites /
Index) • 25 Shards / Index (400K Sites / Shard) • 3 Copies of Data (1 Primary + 2 Replicas) 2 Major Use Cases • Global Search • Query All Shards • Local Search • Query One Shard Indicies & Shards
Nodes & Clusters Warning, YMMV! !
Monitoring Cluster Health
Monitoring Cluster Health
Monitoring Cluster Health
Monitoring Cluster Health
Thanks! @HypertextRanch
[email protected]
xyu