Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Elasticsearch & 63 Million WordPress Sites
Search
xyu
February 06, 2014
Technology
1
200
Elasticsearch & 63 Million WordPress Sites
Overview of the Elasticsearch infrastructure that Automattic maintains to support WordPress.com.
xyu
February 06, 2014
Tweet
Share
More Decks by xyu
See All by xyu
Improving Search With One Weird Trick
xyu
0
75
Better Performance & Uptime Through Async
xyu
0
85
Dealing With Time Travelers in Analytics
xyu
0
70
Scaling WordPress Queries With Elasticsearch
xyu
1
13k
Scale Out Don't Scale Up
xyu
2
190
Understanding Language and Fixing WP Search
xyu
1
160
Jetpack Related Posts for Power Users
xyu
1
520
WP ❤️ Elasticsearch
xyu
1
350
Cloud Powered WordPress for Everyone
xyu
0
97
Other Decks in Technology
See All in Technology
これまでのキャリアとこれからMLエンジニアとしてどう動くか
masatakashiwagi
0
300
XRミーティング 2024-03-20
1ftseabass
PRO
0
100
GraphQLに入門してみた
chiroruxx
2
120
エンジニア候補者向け資料2024.03.28.pdf
macloud
0
2.9k
生成AI・LLM時代における 機械学習エンジニアとしてのキャリア戦略・開発戦略 / my-career-and-development-strategies-for-ml-engineer-2024
yuya4
3
240
OCI Data Integration技術情報 / ocidi_technical_jp
oracle4engineer
PRO
1
1.5k
やっていきテスト
k6s4i53rx
0
160
Azureコストは水道代/The_47th_Tokyo_Jazug
aeonpeople
3
360
BDD(Cucumber)コミュニティが無料提供しているコンテンツの紹介と現在起きている危機
nihonbuson
4
730
技術イベントはなんとかひねり出す 日経の技術広報の取り組み/techpr3
nishiuma
0
220
OpenTelemetry実践 はじめの一歩
taxin
0
310
大規模なアジャイル開発の現場と技術負債 / Technical Debt
yoshiitaka
20
4k
Featured
See All Featured
10 Git Anti Patterns You Should be Aware of
lemiorhan
644
57k
Writing Fast Ruby
sferik
619
59k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
18
1.7k
The Straight Up "How To Draw Better" Workshop
denniskardys
227
130k
What the flash - Photography Introduction
edds
64
11k
Raft: Consensus for Rubyists
vanstee
130
6.2k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
11
1.4k
Atom: Resistance is Futile
akmur
258
25k
Principles of Awesome APIs and How to Build Them.
keavy
119
16k
The Cost Of JavaScript in 2023
addyosmani
13
3.7k
Typedesign – Prime Four
hannesfritz
36
2k
Large-scale JavaScript Application Architecture
addyosmani
501
110k
Transcript
Elasticsearch & 63 Million WordPress Sites Elasticsearch Boston (Feb. 2014)
Xiao Yu Code Wrangler — Automattic @HypertextRanch
[email protected]
xyu
VaultPress Jetpack Simplenote Akismet Polldaddy Gravatar VideoPress IntenseDebate
Simperium Code Poet Cloudup
Cluster Stats • 63M Sites • 743M Documents • 12TB
Primary + Replicas • 51M Query Ops / Day • 15M Index Ops / Day 2 Major Use Cases • Global Search ! • Local Search Elasticsearch + WordPress.com
Infrastructure Layout Internal API Cache REST API PHP Node 1
Node 2 Cluster A Node 1 Node 2 Node 3 Node n Cluster B Stats
Documents & Types /index/post { blog_id: 123, post_id: 456, title:
"Search!", content: "…", blog: { lang: "en", … }, … } /index/blog { blog_id: 123, url: "www.xyu.io", follower_ids: [ 789, … ], lang: "en", indexable: true, … }
Storage Strategy • Grow Number of Indices (10M Sites /
Index) • 25 Shards / Index (400K Sites / Shard) • 3 Copies of Data (1 Primary + 2 Replicas) 2 Major Use Cases • Global Search • Query All Shards • Local Search • Query One Shard Indicies & Shards
Nodes & Clusters Warning, YMMV! !
Monitoring Cluster Health
Monitoring Cluster Health
Monitoring Cluster Health
Monitoring Cluster Health
Thanks! @HypertextRanch
[email protected]
xyu