What is/isn’t this talk about?
> Not an introduction to Polyglot Persistence
> More about how and why with practical examples
> Looking at it more in terms of developer point of view
Slide 5
Slide 5 text
Polyglot Persistence
“A fancy term meaning using multiple data storage
technologies in a solution.
Slide 6
Slide 6 text
Traditional Look
MySuperShoppingApp
My GIANT RDBMS
Shopping Cart
Slide 7
Slide 7 text
Traditional Look
from https://speakerdeck.com/chrisann/cd-making-it-work-with-windows-and-net
Slide 8
Slide 8 text
Polyglot Persistence Look
MySuperShoppingApp
MongoDB
Elasticsearch
Neo4j
SQL Server
Slide 9
Slide 9 text
Advantages
> Tapping into the strengths of each data storage technology
> One of the key drivers towards componentization component based
architecture
> Sometimes, it’s the only way
Slide 10
Slide 10 text
Drawbacks
> Learning + understanding data storage technology-specific interfaces
> Different needs:
• Performance
• Monitoring
• Backups
• Compliance
Slide 11
Slide 11 text
Living Examples: Foursquare
Slide 12
Slide 12 text
Living Examples: Foursquare
> Mainly on MongoDB, Elasticsearch and Hadoop
> Search, recommendation, social graph, listing, ratings and reviews, venue detail
> More on engineering.foursquare.com
Slide 13
Slide 13 text
Living Examples: Foursquare
Slide 14
Slide 14 text
Living Examples: Zleek
Slide 15
Slide 15 text
Living Examples: Zleek
> MongoDB as the primary source of truth
> Elasticsearch for answering search questions and log data
> Neo4j for recommendation and answering interesting questions
> Future: Redis for caching
Slide 16
Slide 16 text
Living Examples: Zleek - MongoDB
> Primary source of truth
> Simple ID lookups and listing queries
> Changes communicated to RabbitMQ
Slide 17
Slide 17 text
Living Examples: Zleek - Elasticsearch
> Syncing data from MongoDB
> Mainly all search requests
• Full-text search queries
• Range queries
• Geo Location queries
Slide 18
Slide 18 text
Living Examples: Zleek - Neo4j
> Syncing the data from MongoDB
> Questions to answer:
• Which grocery store has the ingredients for a recipe?
• Which recipes can be made with the items in a store?
• How my friends interacted with this recipe?
Slide 19
Slide 19 text
Living Examples: Zleek - Neo4j
Slide 20
Slide 20 text
Living Examples: Zleek – Neo4j
“Which recipes can I cook with the items available in grocery store 1?
Slide 21
Slide 21 text
Living Examples: Zleek - Key Points
> Immutability of the data
> RabbitMQ as main event store
> Be sure you need it
> Measure, measure and measure
Slide 22
Slide 22 text
What about development?
Slide 23
Slide 23 text
What about development?
Slide 24
Slide 24 text
What about development?
Slide 25
Slide 25 text
What about development?
> Docker
> Docker Compose
Slide 26
Slide 26 text
What about development?
Slide 27
Slide 27 text
What about development?
Slide 28
Slide 28 text
Shaping the product
“The way you deal with your data will shape the product.
Slide 29
Slide 29 text
Shaping the product: Foursquare Example
Slide 30
Slide 30 text
Shaping the product: Twitter Example
Slide 31
Slide 31 text
Shaping the product: Twitter Example
Slide 32
Slide 32 text
One last thing...
“Well-tune the requirements, consider the trade offs and
come up with a solid, functional plan to serve your data
instead of blindly getting them out through the same door.
Slide 33
Slide 33 text
Here is more information
> Polyglot Persistence from Martin Fowler
http://martinfowler.com/bliki/PolyglotPersistence.html
> An evolving example:
https://github.com/tugberkugurlu/ModernShopping