Smart, Scalable Content Distribution @OReilly Software Architecture New York

Smart, Scalable Content Distribution @OReilly Software Architecture New York

Separating data from functionality is one of the tricks to speedier applications, since it allows accessing the right content at the right time. Partitioning data cleverly presents several challenges that I have overcome on several different projects, and it requires a combination of architecture, DevOps, development, and testing skills that is pretty unique. Unless you have such a Swiss Army knife on your team (and are not afraid of the bus factor), you’ll want to come listen to how I do it.

I share a solution to content snapshotting, distribution, and caching in a silo-based architecture involving tens of machines, walking you through the problems encountered and solutions I came up with. You’ll leave knowing how to automate content distribution on individual nodes and warm up caches, all while keeping your website permanently online.

84cfe0e14cd3fdf8d1b2ef8223d99619?s=128

Georgiana Gligor

April 04, 2017
Tweet

Transcript

  1. SMART, SCALABLE CONTENT DISTRIBUTION GEORGIANA GLIGOR Tekkie Consulting @gbtekkie

  2. 2 ✤ CMS comparison criteria ✤ CMS team roles ✤

    content modeling
  3. 3 ✤ CMS comparison criteria ✤ CMS team roles ✤

    content modeling Not Today
  4. 4 NOT ADDRESSING TODAY content management systems content delivery networks

    build a one-size-fits-all recipe
  5. 5 WE WILL AIM FOR automate content distribution content partitioning

    high availability
  6. 6 Georgiana Gligor ✤ Geek. Mother. Do-er. ✤ Crafting enterprise

    apps ✤ Large-Scale Application Architect ✤ PhD Student in Systems Engineering @gbtekkie gb@tekkie.ro
  7. 7 awareness interest decision action MARKETING FUNNEL

  8. 8 AGENDA COMPLEXITY DISTRIBUTION SILOS

  9. CONTENT COMPLEXITY

  10. 10 What business are you in? ✤ Collaboration ✤ Transactional

    ✤ Media property ✤ SaaS application
  11. 11 CAPTURING CONTENT SECURITY pen & paper structured data user

    interface non-structured digital documents MANUAL VERSIONING
  12. 12 EXPOSING CONTENT SCALING CONSISTENCY structured structured APIs non-structured user

    interface public-facing UI
  13. 13 TRANSACTIONAL APPLICATION transactional UI PCI/PII COMPLIANCE structured structured APIs

    non-structured internal UI
  14. 14 TYPICAL MODERN NEEDS collaboration UI transactional UI YOUR CONTROL

    internal UI APIs digital files content pen & paper internal UI media UI SaaS services APIs 3RD
 PARTY content
  15. 15 TYPICAL MODERN NEEDS collaboration UI transactional UI digital files

    content pen & paper media UI
  16. 16 PUT CONTENT FIRST transactional UI collaboration UI digital files

    content pen & paper media UI
  17. 17 TRANSACTIONAL CONTENT pricing technicalities availability media VERY RARELY RARELY

    INFREQUENTLY FREQUENTLY
  18. 18 COLLABORATION CONTENT Q & A reviews customer content DEMAND

    SPIKES DEMAND SPIKES DEMAND SPIKES
  19. 19 Software industry is built around anticipating change.

  20. SILOS

  21. 21 Departments inside a company don’t work together. - Silo

    Mentality Not Today
  22. 22 IMPLEMENTING CHANGE

  23. CONTENT SOURCES 23 3rd party API content internal API internal

    API CURRENT STATE load balancer frontend middleware middleware middleware cache frontend
  24. CONTENT SOURCES 24 end-to-end functionality 3rd party API content internal

    API internal API CURRENT STATE
  25. CONTENT SOURCES 25 traffic controller silo 1 silo 3 silo

    2 3rd party API content internal API internal API MEET AND GREET
  26. 26 traffic controller variation A variation C variation B THE

    CATCH CONTENT SOURCES 3rd party API content internal API internal API
  27. 27 traffic controller AWS Google private cloud MULTI - CLOUD

    CONTENT SOURCES 3rd party API content internal API internal API
  28. 28 Phrase Of The Day Each day is an opportunity

    It's one thing to say this phrase each day, another to actually apply it. When you're standing in line at Starbucks, remember that this day has never existed before.
  29. 29 traffic controller code content SILO COMPOSITION content sources

  30. 30 traffic controller SILO’D CONTENT content sources

  31. 31 traffic controller v11.4.7 v12.0.1 v12.0.2 CODE VERSIONING content sources

  32. 32 traffic controller v5.1.9 v11.4.7 v5.2.0 v11.4.7 v6.0.1 v12.0.1 CONTENT

    VERSIONING CONTENT SOURCES 3rd party API transactional internal API
  33. CONTENT DISTRIBUTION

  34. 34 CMS UI CMS TO CAPTURE CONTENT content

  35. 35 content CMS UI APIs import import 3RD PARTY DATA

    IMPORT 3rd party DIFFERENT STRUCTURE
  36. 36 CONTENT NORMALIZATION

  37. 37 content import APIs mix & match import mix &

    match 3RD PARTY CONTENT 3rd party
  38. 38 content import APIs mix & match import mix &

    match CMS UI manually resolve data conflicts 3RD PARTY CONTENT 3rd party
  39. 39 content PARTITIONING rarely frequently

  40. 40 content rarely frequently CONTENT AS CODE

  41. 41 content CONTENT AS CODE rarely frequently

  42. AUTOMATED DISTRIBUTION 42 content traffic controller CMS UI

  43. ONE MORE THING…

  44. 44 2 3 EVENTUAL CONSISTENCY

  45. CONTENT SOURCES 45 traffic controller CONSISTENCY MODELS transactional non-transactional

  46. 46 Men have become the tools of their tools. -

    Henry David Thoreau
  47. ‹#› THANK YOU GEORGIANA GLIGOR } Affordable quality system architecture.

    In-house polyglot trainings. gb@tekkie.ro
  48. 48 traffic controller TOO FREQUENTLY UPDATED content

  49. 49 traffic controller TOO FREQUENTLY UPDATED message queues content