Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Surviving Data in Large Doses
Search
Tareq Abedrabbo
November 20, 2013
Technology
250
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Surviving Data in Large Doses
NoSQL Search Roadshow London 2013
Tareq Abedrabbo
November 20, 2013
Transcript
47 slides
Surviving Data in Large Doses Tareq Abedrabbo NoSQL Search Roadshow
London 2013
About me • CTO at OpenCredo • Delivering large-scale data
projects in a number of domains • Co-author of Neo4j in Action (Manning)
What this talk is about…
Supermarkets
Meanwhile, in DevLand
Bob is an application developer
Bob wants to build an application. Bob knows that a
relational database is definitely not the right choice for his application
Bob chooses a NoSQL database because he likes it (he
secretly thinks it’s good for his CV too).
Bob goes for a three-tier architecture. It’s separation of concerns.
It’s best practice.
Bob builds an object model first. It’s Domain Driven Design.
It’s best practice.
Bob uses an object mapping framework. Databases should be hidden
behind layers of abstraction. It’s best practice.
Bob hopes for the best!
What challenges is Bob facing?
Suitability of the data model
Suitability of the architecture and the implementation
Ability to meet new requirements
Being able to use the selected technology to the best
of its ability
Performance
A number of applications built on top of NoSQL technologies
end up unfit for purpose
How did we get ourselves into such a mess?
• Technical evangelism • Evolution in requirements • Unthinking decisions
• Ill-informed opinions
Common problem: there is focus on technology and implementation, not
on real value
So what’s the alternative?
Separation of concerns based on data flow
Data flow
• Lifecycle • Structure • Size • Velocity • Purpose
How?
Identify the concerns: what do I care about?
Identify the locality of these concerns: where are the natural
boundaries?
Build focused specialised models
Compose the models into a complete system
Computing is data structures + algorithms
If we accept that separation of concerns should be applied
to algorithms, it is appropriate to apply the same thinking to data
The real value of this form of separation of concerns
is true decoupling
What’s out there
CQRS
Polyglot Persistence
How do I apply it?
It depends on the data flow :)
For general-purpose data platforms, micro services work well
Build micro services that are closer to the natural underlying
model
Other strategies are possible, for example if the data is
highly volatile, consider in-memory grids
There are practical considerations - obviously
Don’t start with 10 different databases because you think you
might eventually need all of them
How would that impact support and operations?
There is potential for simplification based on clearly targeted usage
Links • Twitter: @tareq_abedrabbo • Blog: http://www.terminalstate.net • OpenCredo: http://www.opencredo.com
Thank you!
More Decks by Tareq Abedrabbo
See All by Tareq Abedrabbo
Not a SO(A) Trivial Question!
tareqabedrabbo
0
95
Designing APIs for Data Driven Systems
tareqabedrabbo
0
79
Things I wish I'd known before I started with Microservices
tareqabedrabbo
0
700
Building a Scalable Event Service with Cassandra: Design to Code
tareqabedrabbo
1
510
The 7 Deadly Sins of Microservices
tareqabedrabbo
7
1.3k
The Ubiquitous Graph
tareqabedrabbo
0
250
The 7 Deadly Sins of Microservices
tareqabedrabbo
0
670
Building a Scalable Event Service with Cassandra: Design to Code
tareqabedrabbo
0
130
Time Series and Events: Storage and Querying Strategies with Cassandra
tareqabedrabbo
0
360
Other Decks in Technology
See All in Technology
ラクスル会社紹介資料|エンジニア向け / RAKSUL Introduction for Engineers
raksulrecruiting
0
130
[関西Kaggler会 20261113] 続・コミュニティコンペのススメ
nejumi
0
120
20261113_JAWS_Osaka
takuyay0ne
0
570
少ない人手 限られた予算 それでも守る 消せない・見つける・戻れる オールハザード型BCP on AWS / 20261107 Mitsutoshi Matsuo
shift_evolve
PRO
0
400
お風呂場をサーバルームにしよう
yusuke427
0
290
BALLAS Technology Deck
ballas_inc
0
780
コピーオンライト型ファイルシステムと ランダム書き込み
sat
PRO
2
120
農家は秘伝のタレを知らない - NOBODY KNOWS THE HIDDEN SOURCE - Kaggriculture 3rd Place Solution
masspeaks
2
490
自動計装なしでECSサービスをApplication Signalsに載せるまで
ota1022
1
170
高品質と高スピードを支える品質のシフトレフトのアプローチ
goyoki
4
1.5k
専門家に使ってもらうAIプロダクトをつくる - 病院の医事課向けDPCコーディング支援システムの設計と評価
hagino3000
2
180
これからのデータエンジニアの役割について考える / Rethinking the Role of Data Engineers in the AI Era
okiyuki99
1
530
Featured
See All Featured
B2B Lead Gen: Tactics, Traps & Triumph
marketingsoph
1
300
Noah Learner - AI + Me: how we built a GSC Bulk Export data pipeline
techseoconnect
PRO
0
490
Visual Storytelling: How to be a Superhuman Communicator
reverentgeek
2
750
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
46
3.1k
sira's awesome portfolio website redesign presentation
elsirapls
0
510
How Fast Is Fast Enough? [PerfNow 2025]
tammyeverts
3
980
[SF Ruby Conf 2025] Rails X
palkan
3
1.5k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4.6k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
38
3.1k
We Are The Robots
honzajavorek
0
450
Max Prin - Stacking Signals: How International SEO Comes Together (And Falls Apart)
techseoconnect
PRO
0
530
Art, The Web, and Tiny UX
lynnandtonic
304
22k