Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Evolution of a Real-Time Web Analytics Platform
Search
Geoff Wagstaff
October 18, 2013
Technology
1
360
Evolution of a Real-Time Web Analytics Platform
Talk about data stores in use at GoSquared at the AllYourBase conference.
Geoff Wagstaff
October 18, 2013
Tweet
Share
More Decks by Geoff Wagstaff
See All by Geoff Wagstaff
GoSquared Presentation at AWS for Startups
thedeveloper
1
650
Other Decks in Technology
See All in Technology
LTに影響を受けてテンプレリポジトリを作った話
hol1kgmg
0
290
AWS re:Inforce 2025 re:Cap Update Pickup & AWS Control Tower の運用における考慮ポイント
htan
1
210
データエンジニアがクラシルでやりたいことの現在地
gappy50
3
850
GMOペパボのデータ基盤とデータ活用の現在地 / Current State of GMO Pepabo's Data Infrastructure and Data Utilization
zaimy
3
200
alecthomas/kong はいいぞ
fujiwara3
6
1.4k
MCP認可の現在地と自律型エージェント対応に向けた課題 / MCP Authorization Today and Challenges to Support Autonomous Agents
yokawasa
5
1.8k
Amazon Bedrock AgentCoreのフロントエンドを探す旅 (Next.js編)
kmiya84377
1
100
Lambda management with ecspresso and Terraform
ijin
2
130
【CEDEC2025】現場を理解して実現!ゲーム開発を効率化するWebサービスの開発と、利用促進のための継続的な改善
cygames
PRO
0
720
ロールが細分化された組織でSREと協働するインフラエンジニアは何をするか? / SRE Lounge #18
kossykinto
0
170
AI関数が早くなったので試してみよう
kumakura
0
120
専門分化が進む分業下でもユーザーが本当に欲しかったものを追求するプロダクトマネジメント/Focus on real user needs despite deep specialization and division of labor
moriyuya
0
1k
Featured
See All Featured
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
130
19k
Testing 201, or: Great Expectations
jmmastey
45
7.6k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
How STYLIGHT went responsive
nonsquared
100
5.7k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
1k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.5k
A designer walks into a library…
pauljervisheath
207
24k
Unsuck your backbone
ammeep
671
58k
Navigating Team Friction
lara
188
15k
Code Reviewing Like a Champion
maltzj
524
40k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Transcript
The Evolution of a Real-Time Analytics Platform Geoff Wagstaff @TheDeveloper
The Now dashboard
The Trends dashboard
Building Real-Time Analytics Behind the “Now” dashboard
Back in 2009 1 server LAMP stack Conventional hosting
LiveStats v1
None
Meltdown!
Problem? First taste of scale WRITES
Reads are easy to scale Primary Writes Replica 1 Replica
2 Replica 3 Reads Reads Reads
Writes? Not so much. Primary MANY WRITES! Replica 1 Replica
2 Replica 3 Reads Reads Reads :(
Scale Horizontally
Node Node Node Requests Requests Requests NginX -> PHP-FPM <-->
Memcache
Problems
Stupidly high data transfer: several TB per day DB ->
app -> DB round trips High latency on DB ops Race conditions
Redis to the rescue! “Advanced in-memory key-value store”
Rich Data types
Rich Data types Keys Hashes Lists Sets Sorted Sets GET
SET HGET HSET HMSET LPUSH LPOP BLPOP SADD SREM SRANGE ZADD ZREM ZRANGE ZINTERSTORE
Distributed locks Service Service Service Fast counters Fan-out Pub/Sub broadcast
Message queues redis-1 redis-2 Solved concurrency problems
ACID
A C I D tomic onsistent solated urable MySQL MongoDB
Other ACID DBs:
Fast
Fast Redis 2.6.16 on 2.4GHz i7 MBP
Single-process, one per core Run on m1.medium - 1 core,
3.5GB memory Redis cluster is coming! Now on Elasticache Redis deployment
Behind the “Trends” dashboard Building Historical Analytics
Trends v1
Sharded MySQL from outset Aging Unreliable Trends v1
The Trends dashboard
MongoDB vs Cassandra
MongoDB Document store: no schema, flexible Compelling replication & sharding
features Fast in-place field updates similar to Redis
Attempt #1: Store & aggregate Document for each list item,
timestamp and site Aggregation framework: match, group, sort Collection per list type Flexible Made app simpler Huge number of documents Slow aggregate queries: ~1s+ ✔ ✔ X X
Attempt #2 Document per list, timestamp and site Collection per
list type Faster lookups (no aggregation) Fewer documents Smaller _id Document size limit Unordered High data transfer ✔ ✔ ✔ X X X
MongoStat
Downsides High random I/O Document size & relocation Fragmentation Database
lock
K.O. MongoDB
Cassandra Distributed hash ring: masterless Linear scalability Built for scale
+ write throughput
CQL
CQL SELECT sql AS cql FROM mysql WHERE query_language =
“good” Not as scary as Column Families + Thrift SQL Schemas + Querying
CQL CREATE TABLE d_aggregate_day ( sid int, ts int, s
text, v counter PRIMARY KEY (sid, ts, s)) partition key cluster key Distributed counters!
B ASE
B A S E asically vailable oft-state ventually consistent
Eventual consistency isn’t a problem More efficient with the disk
Low maintenance Cheap
Redis + Cassandra = win Redis as a speed layer
+ aggregator for lists Cassandra as timeseries counter storage Collector Redis Cassandra Periodic flushes to Cassandra
Exploit DBs strengths Build an indestructible service Use the best
tools for the job
Thanks! Geoff Wagstaff @TheDeveloper engineering.gosquared.com