Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Do you really need that relational DB?

Do you really need that relational DB?

Say you're working with a big e-commerce project. From early days - you knew that normalised DB is the way to go and that's what you do nowadays. One day, the marketing department makes an amazing deal and your website is swarmed by thousands and thousands of customers! That's awesome! Except your DB is sweating so hard that in 2 minutes it releases the last breath and your caches expire. Now what?!

I had this crazy idea, deep in my mind, which no sane person believed in - perhaps we don't really need to query all those relations all the time? Perhaps there is a different way of modelling data for high load? If caching is so hard, are we doing it correctly? I'd like to share this idea with you, with some practical examples from production and theoretical guesses. My goal is to seed new ideas into your minds, feed them with a different approach which you might implement in the future. I'll be talking about data projections, events, queues, data modelling, and de-normalisation. Might mention a buzzword or two like NoSQL, AWS or DDD.

More Decks by Donatas Aleksandravičius

Other Decks in Programming

Transcript

  1. Do you really need that
    relational DB?

    View full-size slide

  2. Do you begin
    coding from DB
    structure?

    View full-size slide

  3. Do you begin
    coding from user's
    perspective?

    View full-size slide

  4. Why do we constantly select
    data that doesn't change?
    https://commons.wikimedia.org/wiki/File:Man-scratching-head.gif

    View full-size slide

  5. Reading same content from
    same source for each view. Why?

    View full-size slide

  6. It's all in the roots

    View full-size slide

  7. Context: Medium - High traffic

    View full-size slide

  8. Perceived speed is important
    • Pinterest increased search engine traffic and sign-ups by 15% when they
    reduced perceived wait times by 40%.

    • COOK increased conversions by 7%, decreased bounce rates by 7%, and
    increased pages per session by 10% when they reduced average page
    load time by 850ms.

    • Source: https://developers.google.com/web/fundamentals/performance/
    why-performance-matters/

    View full-size slide

  9. RDBMS is slow - bbc.com

    View full-size slide

  10. RDBMS is slow - coolblue.nl

    View full-size slide

  11. How to stop relying on RDBMS
    in User facing apps?

    View full-size slide

  12. Typical news portal

    View full-size slide

  13. Multiple slices of same data

    View full-size slide

  14. Multiple slices of same data

    View full-size slide

  15. Multiple slices of same data

    View full-size slide

  16. Multiple slices of same data

    View full-size slide

  17. Multiple slices of same data

    View full-size slide

  18. But it can be cached?

    View full-size slide

  19. But it can be cached?
    • Difficult to determine how long

    View full-size slide

  20. But it can be cached?
    • Difficult to determine how long
    • Big news - short cache

    View full-size slide

  21. But it can be cached?
    • Difficult to determine how long
    • Big news - short cache
    • Regular news - long cache

    View full-size slide

  22. But it can be cached?
    • Difficult to determine how long
    • Big news - short cache
    • Regular news - long cache
    • Too small TTL - more hardware needed

    View full-size slide

  23. But it can be cached?
    • Difficult to determine how long
    • Big news - short cache
    • Regular news - long cache
    • Too small TTL - more hardware needed
    • Too big TTL - the news are outdated of include mistakes

    View full-size slide

  24. No control over cached data
    https://commons.wikimedia.org/wiki/File:Emblem-evil-computer.svg

    View full-size slide

  25. Multiple users generating same
    cache because it expired

    View full-size slide

  26. Caching didn't save my project

    View full-size slide

  27. Static HTML file saved it

    View full-size slide

  28. So what else?

    View full-size slide

  29. Read model?
    • Read model in CQRS

    View full-size slide

  30. Read model?
    • Read model in CQRS
    • Aggregate root in DDD

    View full-size slide

  31. Read model?
    • Read model in CQRS
    • Aggregate root in DDD
    • A view in RDBMS

    View full-size slide

  32. A dataset crafted specifically for
    a single use case on the website

    View full-size slide

  33. You don't need RDBMS for a
    news portal

    View full-size slide

  34. Create seams

    View full-size slide

  35. Projector?
    • Comes from Event Sourcing

    View full-size slide

  36. Projector?
    • Comes from Event Sourcing
    • Responsible to project an event stream to any structural
    representation

    View full-size slide

  37. The Projector

    View full-size slide

  38. The Projector

    View full-size slide

  39. Projected Read Model

    View full-size slide

  40. No need for Event Sourcing

    View full-size slide

  41. Unlimited possibilities

    View full-size slide

  42. Unlimited possibilities
    • Convert straight to HTML

    View full-size slide

  43. Unlimited possibilities
    • Convert straight to HTML
    • Add a queue for big amounts of data

    View full-size slide

  44. Unlimited possibilities
    • Convert straight to HTML
    • Add a queue for big amounts of data
    • Hook up external services for integrations

    View full-size slide

  45. Unlimited possibilities
    • Convert straight to HTML
    • Add a queue for big amounts of data
    • Hook up external services for integrations
    • Project as many different formats as you want

    View full-size slide

  46. But...
    • Costly mistakes - a need to regenerate everything

    View full-size slide

  47. But...
    • Costly mistakes - a need to regenerate everything
    • A lot more moving parts, more complexity

    View full-size slide

  48. But...
    • Costly mistakes - a need to regenerate everything
    • A lot more moving parts, more complexity
    • Sounds a bit "backwards" and unusual

    View full-size slide

  49. How about a harder example?

    View full-size slide

  50. Optimisation for back office

    View full-size slide

  51. Optimisation for back office
    • Analysts will want every possible slice of your data

    View full-size slide

  52. Optimisation for back office
    • Analysts will want every possible slice of your data
    • Very different queries from what you use on production

    View full-size slide

  53. Optimisation for back office
    • Analysts will want every possible slice of your data
    • Very different queries from what you use on production
    • Difficult to find correct indexes to satisfy both worlds

    View full-size slide

  54. Cache
    • Product?

    View full-size slide

  55. Cache
    • Product?
    • Lists?

    View full-size slide

  56. Cache
    • Product?
    • Lists?
    • Both?

    View full-size slide

  57. Cache
    • Product?
    • Lists?
    • Both?
    • Parts of the product?

    View full-size slide

  58. Cache
    • Product?
    • Lists?
    • Both?
    • Parts of the product?
    • Price?

    View full-size slide

  59. Cache
    • Product?
    • Lists?
    • Both?
    • Parts of the product?
    • Price?
    • Users regenerating it on random TTL?

    View full-size slide

  60. You don't need RDBMS to
    display a product page

    View full-size slide

  61. Multiple Read Models for
    similar data
    Storage is Cheap!

    View full-size slide

  62. Be careful of inconsistencies

    View full-size slide

  63. Dealing with lots of data

    View full-size slide

  64. NoSQL streams to the help

    View full-size slide

  65. NoSQL streams to the help
    • Store final result into NoSQL

    View full-size slide

  66. NoSQL streams to the help
    • Store final result into NoSQL
    • Streams trigger the projections

    View full-size slide

  67. NoSQL streams to the help
    • Store final result into NoSQL
    • Streams trigger the projections
    • AWS DynamoDB

    View full-size slide

  68. NoSQL streams to the help
    • Store final result into NoSQL
    • Streams trigger the projections
    • AWS DynamoDB
    • MongoDB Change Streams

    View full-size slide

  69. NoSQL streams to the help
    • Store final result into NoSQL
    • Streams trigger the projections
    • AWS DynamoDB
    • MongoDB Change Streams
    • Something else?

    View full-size slide

  70. Queues to the help
    • AWS SQS

    • RabbitMQ

    • Kafka

    View full-size slide

  71. No injection points?

    View full-size slide

  72. No injection points?
    • RDBMS triggers?

    View full-size slide

  73. No injection points?
    • RDBMS triggers?
    • Simple microservice that just polls DB

    View full-size slide

  74. No injection points?
    • RDBMS triggers?
    • Simple microservice that just polls DB
    • Slave replication

    View full-size slide

  75. No injection points?
    • RDBMS triggers?
    • Simple microservice that just polls DB
    • Slave replication
    • Webhook

    View full-size slide

  76. No injection points?
    • RDBMS triggers?
    • Simple microservice that just polls DB
    • Slave replication
    • Webhook
    • Anything that can trigger your projector

    View full-size slide

  77. Always start from the use case

    View full-size slide

  78. Read Model + Projector = No Cache

    View full-size slide

  79. Read Model + Projector = No Cache
    No Cache = Easier & Longer life

    View full-size slide

  80. You don't need a relational
    database*

    View full-size slide

  81. You don't need a relational
    database*
    * for user facing content

    View full-size slide

  82. Questions?
    Did you like the talk? 

    Was it crap?
    I wanna know, please share your feedback!
    https://joind.in/talk/d687d

    View full-size slide