Architecting Polyglot-Persistent Solutions

Architecting Polyglot-Persistent Solutions

DevConf 2016 (Johannesburg, South Africa)

3a53a5136f530c121a196914aecf3936?s=128

Tugberk Ugurlu

March 08, 2016
Tweet

Transcript

  1. 4.

    What is/isn’t this talk about? > Not an introduction to

    Polyglot Persistence > More about how and why with practical examples > Looking at it more in terms of developer point of view
  2. 9.

    Advantages > Tapping into the strengths of each data storage

    technology > One of the key drivers towards componentization component based architecture > Sometimes, it’s the only way
  3. 10.

    Drawbacks > Learning + understanding data storage technology-specific interfaces >

    Different needs: • Performance • Monitoring • Backups • Compliance
  4. 12.

    Living Examples: Foursquare > Mainly on MongoDB, Elasticsearch and Hadoop

    > Search, recommendation, social graph, listing, ratings and reviews, venue detail > More on engineering.foursquare.com
  5. 15.

    Living Examples: Zleek > MongoDB as the primary source of

    truth > Elasticsearch for answering search questions and log data > Neo4j for recommendation and answering interesting questions > Future: Redis for caching
  6. 16.

    Living Examples: Zleek - MongoDB > Primary source of truth

    > Simple ID lookups and listing queries > Changes communicated to RabbitMQ
  7. 17.

    Living Examples: Zleek - Elasticsearch > Syncing data from MongoDB

    > Mainly all search requests • Full-text search queries • Range queries • Geo Location queries
  8. 18.

    Living Examples: Zleek - Neo4j > Syncing the data from

    MongoDB > Questions to answer: • Which grocery store has the ingredients for a recipe? • Which recipes can be made with the items in a store? • How my friends interacted with this recipe?
  9. 20.

    Living Examples: Zleek – Neo4j “Which recipes can I cook

    with the items available in grocery store 1?
  10. 21.

    Living Examples: Zleek - Key Points > Immutability of the

    data > RabbitMQ as main event store > Be sure you need it > Measure, measure and measure
  11. 32.

    One last thing... “Well-tune the requirements, consider the trade offs

    and come up with a solid, functional plan to serve your data instead of blindly getting them out through the same door.
  12. 33.

    Here is more information > Polyglot Persistence from Martin Fowler

    http://martinfowler.com/bliki/PolyglotPersistence.html > An evolving example: https://github.com/tugberkugurlu/ModernShopping