Slide 1

Slide 1 text

TUGBERK UGURLU Architecting Polyglot-Persistent Solutions Persistence & Data

Slide 2

Slide 2 text

@tourismgeek tugberkugurlu.com github.com/tugberkugurlu stackoverflow.com/users/463785/tugberk

Slide 3

Slide 3 text

@redgate @RedGateCareers red-gate.com/careers

Slide 4

Slide 4 text

What is/isn’t this talk about? > Not an introduction to Polyglot Persistence > More about how and why with practical examples > Looking at it more in terms of developer point of view

Slide 5

Slide 5 text

Polyglot Persistence “A fancy term meaning using multiple data storage technologies in a solution.

Slide 6

Slide 6 text

Traditional Look MySuperShoppingApp My GIANT RDBMS Shopping Cart

Slide 7

Slide 7 text

Traditional Look from https://speakerdeck.com/chrisann/cd-making-it-work-with-windows-and-net

Slide 8

Slide 8 text

Polyglot Persistence Look MySuperShoppingApp MongoDB Elasticsearch Neo4j SQL Server

Slide 9

Slide 9 text

Advantages > Tapping into the strengths of each data storage technology > One of the key drivers towards componentization component based architecture > Sometimes, it’s the only way

Slide 10

Slide 10 text

Drawbacks > Learning + understanding data storage technology-specific interfaces > Different needs: • Performance • Monitoring • Backups • Compliance

Slide 11

Slide 11 text

Living Examples: Foursquare

Slide 12

Slide 12 text

Living Examples: Foursquare > Mainly on MongoDB, Elasticsearch and Hadoop > Search, recommendation, social graph, listing, ratings and reviews, venue detail > More on engineering.foursquare.com

Slide 13

Slide 13 text

Living Examples: Foursquare

Slide 14

Slide 14 text

Living Examples: Zleek

Slide 15

Slide 15 text

Living Examples: Zleek > MongoDB as the primary source of truth > Elasticsearch for answering search questions and log data > Neo4j for recommendation and answering interesting questions > Future: Redis for caching

Slide 16

Slide 16 text

Living Examples: Zleek - MongoDB > Primary source of truth > Simple ID lookups and listing queries > Changes communicated to RabbitMQ

Slide 17

Slide 17 text

Living Examples: Zleek - Elasticsearch > Syncing data from MongoDB > Mainly all search requests • Full-text search queries • Range queries • Geo Location queries

Slide 18

Slide 18 text

Living Examples: Zleek - Neo4j > Syncing the data from MongoDB > Questions to answer: • Which grocery store has the ingredients for a recipe? • Which recipes can be made with the items in a store? • How my friends interacted with this recipe?

Slide 19

Slide 19 text

Living Examples: Zleek - Neo4j

Slide 20

Slide 20 text

Living Examples: Zleek – Neo4j “Which recipes can I cook with the items available in grocery store 1?

Slide 21

Slide 21 text

Living Examples: Zleek - Key Points > Immutability of the data > RabbitMQ as main event store > Be sure you need it > Measure, measure and measure

Slide 22

Slide 22 text

What about development?

Slide 23

Slide 23 text

What about development?

Slide 24

Slide 24 text

What about development?

Slide 25

Slide 25 text

What about development? > Docker > Docker Compose

Slide 26

Slide 26 text

What about development?

Slide 27

Slide 27 text

What about development?

Slide 28

Slide 28 text

Shaping the product “The way you deal with your data will shape the product.

Slide 29

Slide 29 text

Shaping the product: Foursquare Example

Slide 30

Slide 30 text

Shaping the product: Twitter Example

Slide 31

Slide 31 text

Shaping the product: Twitter Example

Slide 32

Slide 32 text

One last thing... “Well-tune the requirements, consider the trade offs and come up with a solid, functional plan to serve your data instead of blindly getting them out through the same door.

Slide 33

Slide 33 text

Here is more information > Polyglot Persistence from Martin Fowler http://martinfowler.com/bliki/PolyglotPersistence.html > An evolving example: https://github.com/tugberkugurlu/ModernShopping

Slide 34

Slide 34 text

/* THANK YOU*/ TUGBERK UGURLU Redgate Software tugberkugurlu.com @tourismgeek http://www.devconf.co.za/