Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Do you really need that relational DB?

Do you really need that relational DB?

Say you're working with a big e-commerce project. From early days - you knew that normalised DB is the way to go and that's what you do nowadays. One day, the marketing department makes an amazing deal and your website is swarmed by thousands and thousands of customers! That's awesome! Except your DB is sweating so hard that in 2 minutes it releases the last breath and your caches expire. Now what?!

I had this crazy idea, deep in my mind, which no sane person believed in - perhaps we don't really need to query all those relations all the time? Perhaps there is a different way of modelling data for high load? If caching is so hard, are we doing it correctly? I'd like to share this idea with you, with some practical examples from production and theoretical guesses. My goal is to seed new ideas into your minds, feed them with a different approach which you might implement in the future. I'll be talking about data projections, events, queues, data modelling, and de-normalisation. Might mention a buzzword or two like NoSQL, AWS or DDD.

More Decks by Donatas Aleksandravičius

Other Decks in Programming

Transcript

  1. Perceived speed is important • Pinterest increased search engine traffic

    and sign-ups by 15% when they reduced perceived wait times by 40%. • COOK increased conversions by 7%, decreased bounce rates by 7%, and increased pages per session by 10% when they reduced average page load time by 850ms. • Source: https://developers.google.com/web/fundamentals/performance/ why-performance-matters/
  2. But it can be cached? • Difficult to determine how

    long • Big news - short cache • Regular news - long cache
  3. But it can be cached? • Difficult to determine how

    long • Big news - short cache • Regular news - long cache • Too small TTL - more hardware needed
  4. But it can be cached? • Difficult to determine how

    long • Big news - short cache • Regular news - long cache • Too small TTL - more hardware needed • Too big TTL - the news are outdated of include mistakes
  5. Projector? • Comes from Event Sourcing • Responsible to project

    an event stream to any structural representation
  6. Unlimited possibilities • Convert straight to HTML • Add a

    queue for big amounts of data • Hook up external services for integrations
  7. Unlimited possibilities • Convert straight to HTML • Add a

    queue for big amounts of data • Hook up external services for integrations • Project as many different formats as you want
  8. But... • Costly mistakes - a need to regenerate everything

    • A lot more moving parts, more complexity
  9. But... • Costly mistakes - a need to regenerate everything

    • A lot more moving parts, more complexity • Sounds a bit "backwards" and unusual
  10. Optimisation for back office • Analysts will want every possible

    slice of your data • Very different queries from what you use on production
  11. Optimisation for back office • Analysts will want every possible

    slice of your data • Very different queries from what you use on production • Difficult to find correct indexes to satisfy both worlds
  12. Cache • Product? • Lists? • Both? • Parts of

    the product? • Price? • Users regenerating it on random TTL?
  13. NoSQL streams to the help • Store final result into

    NoSQL • Streams trigger the projections
  14. NoSQL streams to the help • Store final result into

    NoSQL • Streams trigger the projections • AWS DynamoDB
  15. NoSQL streams to the help • Store final result into

    NoSQL • Streams trigger the projections • AWS DynamoDB • MongoDB Change Streams
  16. NoSQL streams to the help • Store final result into

    NoSQL • Streams trigger the projections • AWS DynamoDB • MongoDB Change Streams • Something else?
  17. No injection points? • RDBMS triggers? • Simple microservice that

    just polls DB • Slave replication • Webhook
  18. No injection points? • RDBMS triggers? • Simple microservice that

    just polls DB • Slave replication • Webhook • Anything that can trigger your projector
  19. Questions? Did you like the talk? 
 Was it crap?

    I wanna know, please share your feedback! https://joind.in/talk/d687d