Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
NoSQL: Not Only a Fairy Tale
Search
Sebastian Cohnen
May 30, 2012
Technology
14k
4
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
NoSQL: Not Only a Fairy Tale
Talk of Timo Derstappen and me at the NoSQL Matters conference in 2012
Sebastian Cohnen
May 30, 2012
Transcript
37 slides
NoSQL Not only a fairy tale Sebastian Cohnen @tisba tisba.de
Timo Derstappen @teemow adcloud.com http://en.wikipedia.org/wiki/File:Old_book_-_Timeless_Books.jpg
Preface
Terms • placement & ads • ad priority
System Overview • administrative back office • worker queue •
almost no NoSQL • serving ads • tracking • here be NoSQLs! platform adserver publishing ads & placements stats & tracking data
Once upon a time… …way back in 2008
Simple Storage Service
Publishing to S3 • gather ad & placement data •
add some JavaScript • publish everything to S3
Ad Delivery via S3 • user visits a website •
deliver JavaScript via CDN • choose and display ads
but, • publishing to S3 was rather expensive • no
incremental update of denormalized data
The relaxed Knight …came along in 2009
CouchDB • REST & JavaScript? nice! • M/R Views •
Multi-Master setup platform adserver adserver adserver
CouchDB only • normalize the data (a bit) • split
by update frequency • BUT… n-m relations are hard to model • and persistent, incremental views are rather useless to us
:-(
CouchDB + node.js • use node.js to assemble data (n-m
relation) • cache response using nginx • also cache some data in node.js
Request flow • incoming request • nginx cache miss •
fetch placement & priorities • process data & fetch ads • send response
How to monitor Consistency? • write tracer documents • measure
replication delay
Achievements • reduced turnaround for publishing priorities by >50% •
build foundation for new features
New Feature Requests …ahead in early 2011
The Problem • requests eventually are going to be unique
• therefor less requests can be cached • CouchDB too slow for our needs • caching things within a node.js process was a bad idea too
Redis • during a cache warmup phase we pre-fill redis
with placement and ad data • all live request are served out of redis • data is updated in the background
…in late 2011 Scalability
How we used CouchDB • >10k updates/h • single source
of changes • multi-master replication • append-only • durability • MVCC usage not required
Resulting Issues • problems with replication and high load •
more instances, more replication, even more load • compaction was a pain too
Whose fault? • not only CouchDB’s fault • simply the
wrong use case • one source for updates • no need for append-only reliability
What now?
Back to S3! • with Redis caching in place… •
move placement and ad data to S3 • cache warming upfront and background updates work just fine!
S3 vs CouchDB • S3 simply fits our needs •
no need to implement sync checks or run compaction • fewer moving parts • less state on our application servers
Once again, more features …ahead in early 2012
Status Quo • first S3-based “adserver” did the ad selection
on the client side • to a certain degree this is still the case
The Challenge • prepare the systems for Real-time bidding •
enable the adserver to decide ad selection server-side • do it fast, say within 25ms or less
Remember Redis? • we know and trust Redis’ performance •
it has sorted sets • we have sets of ads to display for a placement Eureka!
Redis Reloaded! • heavily use sorted sets • create sets
of ads… • we can choose from • which cannot be displayed at all • use ZUNIONSTORE & ZRANGEBYSCORE to precisely select ads
Redis Reloaded! • Redis became a deeply integrated part of
the core business logic • it was very easy to model our needs with Redis • besides enabling new features, we reduced the response payload by >75%
Conclusion
• try to go as incremental as possible • drivers
for architectural decisions… • features • quality & performance • scalability What worked for us…
The End!
• Questions (if time permits) • Visit us at the
adcloud booth Sebastian Cohnen @tisba tisba.de Timo Derstappen @teemow adcloud.com The End!
More Decks by Sebastian Cohnen
See All by Sebastian Cohnen
The Life of a Load Generator
tisba
0
890
Load Testing with 1M Users
tisba
2
3k
Performance Testing Serverless
tisba
0
200
Performance Testing 101, code.talks commerce 2018 [DE]
tisba
2
470
Why we did not choose Microservices to replace a Legacy System
tisba
1
200
Performance Testing 101 [DE]
tisba
0
170
Load Testing with 1,000,000 Users!
tisba
0
230
code.talks 2016: Last- und Performancetests in der Cloud [DE]
tisba
1
970
FrOSCon 2016: Last- und Performancetests in der Cloud?! [DE]
tisba
0
400
Other Decks in Technology
See All in Technology
gem install とrequire のむこうがわ
ydah
0
240
統制されたID・権限をどう利用するか : DPoP/FAPIによるAPIセキュリティ強化
oidfj
1
230
Amazon Bedrock AgentCore の基礎とセキュリティ・ガバナンス
ren8k
0
200
Datadog の権限統制 〜Org 統合に向けて調べた注意点 / JDDUG-Meetup_24
visional_engineering_and_design
1
200
個人ASを取得して、自宅Kubernetes上のVMをグローバルIPで公開した話
munenick
0
210
Re:ゼロから始めるシェアハウスのペットな鯖
silmin_
0
260
設計手法を「道具」として使いこなすために
a_suenami
7
1.4k
要件定義をAIに委任!Claude Codeプラグインで作るPdM × AgentTeams
puku0x
0
110
Kubernetes, Agones, WebRTC, GPU で作る クラウドゲーミング基盤を本番運用に乗せるまで
vivion
0
190
マネージャーをエスパーにしない マネジメントされる側から始める、働きやすい現場づくり
comi190327
1
830
SORACOM × AWS IoT Core でSREする
butadora
0
120
市販IoT機器や自作IoT機器とHomeAssistantを駆使して家の電気代をできる限り見える化する方法の考案
harutiro
0
390
Featured
See All Featured
Building Adaptive Systems
keathley
44
3.3k
Balancing Empowerment & Direction
lara
6
1.4k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
28
3.8k
Faster Mobile Websites
deanohume
310
32k
The Illustrated Guide to Node.js - THAT Conference 2024
reverentgeek
1
610
Optimizing for Happiness
mojombo
378
72k
Why You Should Never Use an ORM
jnunemaker
PRO
61
10k
Code Review Best Practice
trishagee
74
20k
Tips & Tricks on How to Get Your First Job In Tech
honzajavorek
1
850
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
659
62k
Building a A Zero-Code AI SEO Workflow
portentint
PRO
0
840
30 Presentation Tips
portentint
PRO
1
470