Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Do you really need that relational DB?

Do you really need that relational DB?

Say you're working with a big e-commerce project. From early days - you knew that normalised DB is the way to go and that's what you do nowadays. One day, the marketing department makes an amazing deal and your website is swarmed by thousands and thousands of customers! That's awesome! Except your DB is sweating so hard that in 2 minutes it releases the last breath and your caches expire. Now what?!

I had this crazy idea, deep in my mind, which no sane person believed in - perhaps we don't really need to query all those relations all the time? Perhaps there is a different way of modelling data for high load? If caching is so hard, are we doing it correctly? I'd like to share this idea with you, with some practical examples from production and theoretical guesses. My goal is to seed new ideas into your minds, feed them with a different approach which you might implement in the future. I'll be talking about data projections, events, queues, data modelling, and de-normalisation. Might mention a buzzword or two like NoSQL, AWS or DDD.

More Decks by Donatas Aleksandravičius

Other Decks in Programming

Transcript

  1. Do you really need that
    relational DB?

    View Slide

  2. Do you begin
    coding from DB
    structure?

    View Slide

  3. Do you begin
    coding from user's
    perspective?

    View Slide

  4. Why do we constantly select
    data that doesn't change?
    https://commons.wikimedia.org/wiki/File:Man-scratching-head.gif

    View Slide

  5. Reading same content from
    same source for each view. Why?

    View Slide

  6. It's all in the roots

    View Slide

  7. Context: Medium - High traffic

    View Slide

  8. Perceived speed is important
    • Pinterest increased search engine traffic and sign-ups by 15% when they
    reduced perceived wait times by 40%.

    • COOK increased conversions by 7%, decreased bounce rates by 7%, and
    increased pages per session by 10% when they reduced average page
    load time by 850ms.

    • Source: https://developers.google.com/web/fundamentals/performance/
    why-performance-matters/

    View Slide

  9. RDBMS is slow - bbc.com

    View Slide

  10. RDBMS is slow - coolblue.nl

    View Slide

  11. How to stop relying on RDBMS
    in User facing apps?

    View Slide

  12. The setup

    View Slide

  13. Typical news portal

    View Slide

  14. Multiple slices of same data

    View Slide

  15. Multiple slices of same data

    View Slide

  16. Multiple slices of same data

    View Slide

  17. Multiple slices of same data

    View Slide

  18. Multiple slices of same data

    View Slide

  19. But it can be cached?

    View Slide

  20. But it can be cached?
    • Difficult to determine how long

    View Slide

  21. But it can be cached?
    • Difficult to determine how long
    • Big news - short cache

    View Slide

  22. But it can be cached?
    • Difficult to determine how long
    • Big news - short cache
    • Regular news - long cache

    View Slide

  23. But it can be cached?
    • Difficult to determine how long
    • Big news - short cache
    • Regular news - long cache
    • Too small TTL - more hardware needed

    View Slide

  24. But it can be cached?
    • Difficult to determine how long
    • Big news - short cache
    • Regular news - long cache
    • Too small TTL - more hardware needed
    • Too big TTL - the news are outdated of include mistakes

    View Slide

  25. No control over cached data
    https://commons.wikimedia.org/wiki/File:Emblem-evil-computer.svg

    View Slide

  26. Multiple users generating same
    cache because it expired

    View Slide

  27. Caching didn't save my project

    View Slide

  28. Static HTML file saved it

    View Slide

  29. So what else?

    View Slide

  30. View Slide

  31. View Slide

  32. Read Model

    View Slide

  33. Read model?

    View Slide

  34. Read model?
    • Read model in CQRS

    View Slide

  35. Read model?
    • Read model in CQRS
    • Aggregate root in DDD

    View Slide

  36. Read model?
    • Read model in CQRS
    • Aggregate root in DDD
    • A view in RDBMS

    View Slide

  37. A dataset crafted specifically for
    a single use case on the website

    View Slide

  38. View Slide

  39. View Slide

  40. View Slide

  41. View Slide

  42. View Slide

  43. View Slide

  44. View Slide

  45. You don't need RDBMS for a
    news portal

    View Slide

  46. View Slide

  47. Create seams

    View Slide

  48. Projector?

    View Slide

  49. Projector?
    • Comes from Event Sourcing

    View Slide

  50. Projector?
    • Comes from Event Sourcing
    • Responsible to project an event stream to any structural
    representation

    View Slide

  51. The Projector

    View Slide

  52. The Projector

    View Slide

  53. Projected Read Model

    View Slide

  54. No need for Event Sourcing

    View Slide

  55. Unlimited possibilities

    View Slide

  56. Unlimited possibilities
    • Convert straight to HTML

    View Slide

  57. Unlimited possibilities
    • Convert straight to HTML
    • Add a queue for big amounts of data

    View Slide

  58. Unlimited possibilities
    • Convert straight to HTML
    • Add a queue for big amounts of data
    • Hook up external services for integrations

    View Slide

  59. Unlimited possibilities
    • Convert straight to HTML
    • Add a queue for big amounts of data
    • Hook up external services for integrations
    • Project as many different formats as you want

    View Slide

  60. But...

    View Slide

  61. But...
    • Costly mistakes - a need to regenerate everything

    View Slide

  62. But...
    • Costly mistakes - a need to regenerate everything
    • A lot more moving parts, more complexity

    View Slide

  63. But...
    • Costly mistakes - a need to regenerate everything
    • A lot more moving parts, more complexity
    • Sounds a bit "backwards" and unusual

    View Slide

  64. View Slide

  65. View Slide

  66. View Slide

  67. View Slide

  68. View Slide

  69. View Slide

  70. View Slide

  71. View Slide

  72. How about a harder example?

    View Slide

  73. View Slide

  74. View Slide

  75. Optimisation for back office

    View Slide

  76. Optimisation for back office
    • Analysts will want every possible slice of your data

    View Slide

  77. Optimisation for back office
    • Analysts will want every possible slice of your data
    • Very different queries from what you use on production

    View Slide

  78. Optimisation for back office
    • Analysts will want every possible slice of your data
    • Very different queries from what you use on production
    • Difficult to find correct indexes to satisfy both worlds

    View Slide

  79. Cache

    View Slide

  80. Cache
    • Product?

    View Slide

  81. Cache
    • Product?
    • Lists?

    View Slide

  82. Cache
    • Product?
    • Lists?
    • Both?

    View Slide

  83. Cache
    • Product?
    • Lists?
    • Both?
    • Parts of the product?

    View Slide

  84. Cache
    • Product?
    • Lists?
    • Both?
    • Parts of the product?
    • Price?

    View Slide

  85. Cache
    • Product?
    • Lists?
    • Both?
    • Parts of the product?
    • Price?
    • Users regenerating it on random TTL?

    View Slide

  86. You don't need RDBMS to
    display a product page

    View Slide

  87. Multiple Read Models for
    similar data
    Storage is Cheap!

    View Slide

  88. Be careful of inconsistencies

    View Slide

  89. View Slide

  90. View Slide

  91. Dealing with lots of data

    View Slide

  92. NoSQL streams to the help

    View Slide

  93. NoSQL streams to the help
    • Store final result into NoSQL

    View Slide

  94. NoSQL streams to the help
    • Store final result into NoSQL
    • Streams trigger the projections

    View Slide

  95. NoSQL streams to the help
    • Store final result into NoSQL
    • Streams trigger the projections
    • AWS DynamoDB

    View Slide

  96. NoSQL streams to the help
    • Store final result into NoSQL
    • Streams trigger the projections
    • AWS DynamoDB
    • MongoDB Change Streams

    View Slide

  97. NoSQL streams to the help
    • Store final result into NoSQL
    • Streams trigger the projections
    • AWS DynamoDB
    • MongoDB Change Streams
    • Something else?

    View Slide

  98. Queues to the help
    • AWS SQS

    • RabbitMQ

    • Kafka

    View Slide

  99. No injection points?

    View Slide

  100. No injection points?
    • RDBMS triggers?

    View Slide

  101. No injection points?
    • RDBMS triggers?
    • Simple microservice that just polls DB

    View Slide

  102. No injection points?
    • RDBMS triggers?
    • Simple microservice that just polls DB
    • Slave replication

    View Slide

  103. No injection points?
    • RDBMS triggers?
    • Simple microservice that just polls DB
    • Slave replication
    • Webhook

    View Slide

  104. No injection points?
    • RDBMS triggers?
    • Simple microservice that just polls DB
    • Slave replication
    • Webhook
    • Anything that can trigger your projector

    View Slide

  105. View Slide

  106. View Slide

  107. View Slide

  108. View Slide

  109. View Slide

  110. View Slide

  111. Always start from the use case

    View Slide

  112. Read Model + Projector = No Cache

    View Slide

  113. Read Model + Projector = No Cache
    No Cache = Easier & Longer life

    View Slide

  114. You don't need a relational
    database*

    View Slide

  115. You don't need a relational
    database*
    * for user facing content

    View Slide

  116. Questions?
    Did you like the talk? 

    Was it crap?
    I wanna know, please share your feedback!
    https://joind.in/talk/d687d

    View Slide