Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Surviving Data in Large Doses
Search
Tareq Abedrabbo
November 20, 2013
Technology
0
210
Surviving Data in Large Doses
NoSQL Search Roadshow London 2013
Tareq Abedrabbo
November 20, 2013
Tweet
Share
More Decks by Tareq Abedrabbo
See All by Tareq Abedrabbo
Not a SO(A) Trivial Question!
tareqabedrabbo
0
60
Designing APIs for Data Driven Systems
tareqabedrabbo
0
55
Things I wish I'd known before I started with Microservices
tareqabedrabbo
0
660
Building a Scalable Event Service with Cassandra: Design to Code
tareqabedrabbo
1
470
The 7 Deadly Sins of Microservices
tareqabedrabbo
7
1.2k
The Ubiquitous Graph
tareqabedrabbo
0
200
The 7 Deadly Sins of Microservices
tareqabedrabbo
0
610
Building a Scalable Event Service with Cassandra: Design to Code
tareqabedrabbo
0
72
Time Series and Events: Storage and Querying Strategies with Cassandra
tareqabedrabbo
0
290
Other Decks in Technology
See All in Technology
Kiroでインフラ要件定義~テスト を実施してみた
nagisa53
3
310
LLM 機能を支える Langfuse / ClickHouse のサーバレス化
yuu26
6
670
Nx × AI によるモノレポ活用 〜コードジェネレーター編〜
puku0x
0
380
MCP認可の現在地と自律型エージェント対応に向けた課題 / MCP Authorization Today and Challenges to Support Autonomous Agents
yokawasa
5
1.9k
マルチプロダクト×マルチテナントを支えるモジュラモノリスを中心としたアソビューのアーキテクチャ
disc99
1
350
AIのグローバルトレンド 2025 / ai global trend 2025
kyonmm
PRO
1
120
大規模イベントに向けた ABEMA アーキテクチャの遍歴 ~ Platform Strategy 詳細解説 ~
nagapad
0
190
Google Agentspaceを実際に導入した効果と今後の展望
mixi_engineers
PRO
3
340
Amazon Bedrock AgentCoreのフロントエンドを探す旅 (Next.js編)
kmiya84377
1
120
Serverless Meetup #21
yoshidashingo
1
110
Amazon Q Developerを活用したアーキテクチャのリファクタリング
k1nakayama
2
200
専門分化が進む分業下でもユーザーが本当に欲しかったものを追求するプロダクトマネジメント/Focus on real user needs despite deep specialization and division of labor
moriyuya
1
1.1k
Featured
See All Featured
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.6k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
44
2.4k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
60k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Building Flexible Design Systems
yeseniaperezcruz
328
39k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3.1k
Code Reviewing Like a Champion
maltzj
524
40k
The Cult of Friendly URLs
andyhume
79
6.5k
YesSQL, Process and Tooling at Scale
rocio
173
14k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
30
2.2k
Building a Modern Day E-commerce SEO Strategy
aleyda
43
7.4k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
Transcript
Surviving Data in Large Doses Tareq Abedrabbo NoSQL Search Roadshow
London 2013
About me • CTO at OpenCredo • Delivering large-scale data
projects in a number of domains • Co-author of Neo4j in Action (Manning)
What this talk is about…
Supermarkets
Meanwhile, in DevLand
Bob is an application developer
Bob wants to build an application. Bob knows that a
relational database is definitely not the right choice for his application
Bob chooses a NoSQL database because he likes it (he
secretly thinks it’s good for his CV too).
Bob goes for a three-tier architecture. It’s separation of concerns.
It’s best practice.
Bob builds an object model first. It’s Domain Driven Design.
It’s best practice.
Bob uses an object mapping framework. Databases should be hidden
behind layers of abstraction. It’s best practice.
Bob hopes for the best!
What challenges is Bob facing?
Suitability of the data model
Suitability of the architecture and the implementation
Ability to meet new requirements
Being able to use the selected technology to the best
of its ability
Performance
A number of applications built on top of NoSQL technologies
end up unfit for purpose
How did we get ourselves into such a mess?
• Technical evangelism • Evolution in requirements • Unthinking decisions
• Ill-informed opinions
Common problem: there is focus on technology and implementation, not
on real value
So what’s the alternative?
Separation of concerns based on data flow
Data flow
• Lifecycle • Structure • Size • Velocity • Purpose
How?
Identify the concerns: what do I care about?
Identify the locality of these concerns: where are the natural
boundaries?
Build focused specialised models
Compose the models into a complete system
Computing is data structures + algorithms
If we accept that separation of concerns should be applied
to algorithms, it is appropriate to apply the same thinking to data
The real value of this form of separation of concerns
is true decoupling
What’s out there
CQRS
Polyglot Persistence
How do I apply it?
It depends on the data flow :)
For general-purpose data platforms, micro services work well
Build micro services that are closer to the natural underlying
model
Other strategies are possible, for example if the data is
highly volatile, consider in-memory grids
There are practical considerations - obviously
Don’t start with 10 different databases because you think you
might eventually need all of them
How would that impact support and operations?
There is potential for simplification based on clearly targeted usage
Links • Twitter: @tareq_abedrabbo • Blog: http://www.terminalstate.net • OpenCredo: http://www.opencredo.com
Thank you!