Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Architecting Polyglot-Persistent Solutions

Architecting Polyglot-Persistent Solutions

DevConf 2016 (Johannesburg, South Africa)

Tugberk Ugurlu

March 08, 2016
Tweet

More Decks by Tugberk Ugurlu

Other Decks in Programming

Transcript

  1. What is/isn’t this talk about? > Not an introduction to

    Polyglot Persistence > More about how and why with practical examples > Looking at it more in terms of developer point of view
  2. Advantages > Tapping into the strengths of each data storage

    technology > One of the key drivers towards componentization component based architecture > Sometimes, it’s the only way
  3. Drawbacks > Learning + understanding data storage technology-specific interfaces >

    Different needs: • Performance • Monitoring • Backups • Compliance
  4. Living Examples: Foursquare > Mainly on MongoDB, Elasticsearch and Hadoop

    > Search, recommendation, social graph, listing, ratings and reviews, venue detail > More on engineering.foursquare.com
  5. Living Examples: Zleek > MongoDB as the primary source of

    truth > Elasticsearch for answering search questions and log data > Neo4j for recommendation and answering interesting questions > Future: Redis for caching
  6. Living Examples: Zleek - MongoDB > Primary source of truth

    > Simple ID lookups and listing queries > Changes communicated to RabbitMQ
  7. Living Examples: Zleek - Elasticsearch > Syncing data from MongoDB

    > Mainly all search requests • Full-text search queries • Range queries • Geo Location queries
  8. Living Examples: Zleek - Neo4j > Syncing the data from

    MongoDB > Questions to answer: • Which grocery store has the ingredients for a recipe? • Which recipes can be made with the items in a store? • How my friends interacted with this recipe?
  9. Living Examples: Zleek – Neo4j “Which recipes can I cook

    with the items available in grocery store 1?
  10. Living Examples: Zleek - Key Points > Immutability of the

    data > RabbitMQ as main event store > Be sure you need it > Measure, measure and measure
  11. One last thing... “Well-tune the requirements, consider the trade offs

    and come up with a solid, functional plan to serve your data instead of blindly getting them out through the same door.
  12. Here is more information > Polyglot Persistence from Martin Fowler

    http://martinfowler.com/bliki/PolyglotPersistence.html > An evolving example: https://github.com/tugberkugurlu/ModernShopping