$30 off During Our Annual Pro Sale. View details »

Architecting Polyglot-Persistent Solutions

Architecting Polyglot-Persistent Solutions

DevConf 2016 (Johannesburg, South Africa)

Tugberk Ugurlu

March 08, 2016

More Decks by Tugberk Ugurlu

Other Decks in Programming


  1. TUGBERK UGURLU Architecting Polyglot-Persistent Solutions Persistence & Data

  2. @tourismgeek tugberkugurlu.com github.com/tugberkugurlu stackoverflow.com/users/463785/tugberk

  3. @redgate @RedGateCareers red-gate.com/careers

  4. What is/isn’t this talk about? > Not an introduction to

    Polyglot Persistence > More about how and why with practical examples > Looking at it more in terms of developer point of view
  5. Polyglot Persistence “A fancy term meaning using multiple data storage

    technologies in a solution.
  6. Traditional Look MySuperShoppingApp My GIANT RDBMS Shopping Cart

  7. Traditional Look from https://speakerdeck.com/chrisann/cd-making-it-work-with-windows-and-net

  8. Polyglot Persistence Look MySuperShoppingApp MongoDB Elasticsearch Neo4j SQL Server

  9. Advantages > Tapping into the strengths of each data storage

    technology > One of the key drivers towards componentization component based architecture > Sometimes, it’s the only way
  10. Drawbacks > Learning + understanding data storage technology-specific interfaces >

    Different needs: • Performance • Monitoring • Backups • Compliance
  11. Living Examples: Foursquare

  12. Living Examples: Foursquare > Mainly on MongoDB, Elasticsearch and Hadoop

    > Search, recommendation, social graph, listing, ratings and reviews, venue detail > More on engineering.foursquare.com
  13. Living Examples: Foursquare

  14. Living Examples: Zleek

  15. Living Examples: Zleek > MongoDB as the primary source of

    truth > Elasticsearch for answering search questions and log data > Neo4j for recommendation and answering interesting questions > Future: Redis for caching
  16. Living Examples: Zleek - MongoDB > Primary source of truth

    > Simple ID lookups and listing queries > Changes communicated to RabbitMQ
  17. Living Examples: Zleek - Elasticsearch > Syncing data from MongoDB

    > Mainly all search requests • Full-text search queries • Range queries • Geo Location queries
  18. Living Examples: Zleek - Neo4j > Syncing the data from

    MongoDB > Questions to answer: • Which grocery store has the ingredients for a recipe? • Which recipes can be made with the items in a store? • How my friends interacted with this recipe?
  19. Living Examples: Zleek - Neo4j

  20. Living Examples: Zleek – Neo4j “Which recipes can I cook

    with the items available in grocery store 1?
  21. Living Examples: Zleek - Key Points > Immutability of the

    data > RabbitMQ as main event store > Be sure you need it > Measure, measure and measure
  22. What about development?

  23. What about development?

  24. What about development?

  25. What about development? > Docker > Docker Compose

  26. What about development?

  27. What about development?

  28. Shaping the product “The way you deal with your data

    will shape the product.
  29. Shaping the product: Foursquare Example

  30. Shaping the product: Twitter Example

  31. Shaping the product: Twitter Example

  32. One last thing... “Well-tune the requirements, consider the trade offs

    and come up with a solid, functional plan to serve your data instead of blindly getting them out through the same door.
  33. Here is more information > Polyglot Persistence from Martin Fowler

    http://martinfowler.com/bliki/PolyglotPersistence.html > An evolving example: https://github.com/tugberkugurlu/ModernShopping
  34. /* THANK YOU*/ TUGBERK UGURLU Redgate Software tugberkugurlu.com @tourismgeek http://www.devconf.co.za/