Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Beyond The Buzzwords - Microservices and Event-Driven Architecture

Ryan Townsend
September 26, 2018

Beyond The Buzzwords - Microservices and Event-Driven Architecture

Talk given at Dreamforce 2018, September 26th 2018 in Moscone West, San Francisco.

Let's set aside the marketing spin for a moment and focus on the real-world problems that microservices and event streams can solve for us. We'll explore why you should consider these patterns, the exciting technologies that support them, such as Apache Kafka, and most importantly, how you can set aside your fears and be confident in what you're building now and the future.

Ryan Townsend

September 26, 2018
Tweet

More Decks by Ryan Townsend

Other Decks in Programming

Transcript

  1. Beyond the Buzzwords:
    Microservices and Event-Driven Architecture

    with Apache Kafka on Heroku
    Ryan Townsend
    SHIFT Commerce CTO
    Chris Castle
    Heroku Developer Advocate

    View full-size slide

  2. This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the
    assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we
    make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber
    growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief,
    any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services.
    The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new
    products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays
    in our Web hosting, breach of our security measures, the outcome of any litigation, risks associated with completed and any possible mergers and acquisitions, the
    immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth,
    new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger
    enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form
    10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent fiscal quarter. These documents and others containing important
    disclosures are available on the SEC Filings section of the Investor Information section of our Web site.
    Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be
    delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available.
    Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.
    Statement under the Private Securities Litigation Reform Act of 1995
    Forward-Looking Statement

    View full-size slide

  3. Why are we here?
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  4. Anxiety
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  5. Monoliths and Microservices
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  6. “There is no industry consensus yet regarding the properties
    of microservices, and an official definition is missing as well.”

    - Wikipedia
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  7. [email protected]
    Ryan Townsend, CTO
    [email protected]
    Chris Castle, Developer Advocate
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  8. 20 minutes
    No time for questions!
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  9. Why the hype around microservices?
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  10. All the cool kids are doing it
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  11. But what are the real benefits?
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  12. @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2
    Strong Module
    Boundaries
    Independent
    Deployment
    Technology
    Diversity
    Source: Martin Fowler, ThoughtWorks

    View full-size slide

  13. If this isn’t you, that’s ok!
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  14. Because there is a price to pay
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  15. Data / Functionality Distribution
    Eventual Consistency
    Operational Complexity
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2
    Source: Martin Fowler, ThoughtWorks

    View full-size slide

  16. How do we mitigate these?
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  17. Demo Time!
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  18. Solution: Data / Functionality Distribution
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  19. Solution: Data / Functionality Distribution
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2
    n * (n-1) / 2
    Max possible connections

    View full-size slide

  20. Solution: Data / Functionality Distribution
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2
    Source: AWS

    View full-size slide

  21. Solution: Data / Functionality Distribution
    Digital Asset
    Management
    Product
    Information
    Management
    Upload product-123.jpg!
    Admin Panel
    Photographer
    Database or
    Search Index
    Merchandiser
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2
    Chronological Event Stream (Kafka)
    Uploaded Event
    What images can I
    attach to my products?
    Uploaded Event
    Add image to index

    View full-size slide

  22. Solution: Eventual Consistency
    Digital Asset
    Management
    Product
    Information
    Management
    Delete product-123.jpg!
    Admin Panel
    Photographer
    Merchandiser
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2
    Event
    Create Product #123 with
    product-123.jpg
    Event
    Send to trash
    Event
    Untrash
    Admin Panel
    Chronological Event Stream (Kafka)

    View full-size slide

  23. Start with monolith and
    extract pieces – build
    up your maturity
    Solution: Operational Complexity
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  24. Cost of engineers vs
    cost of Heroku
    Solution: Operational Complexity
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2
    Opportunity Cost!

    View full-size slide

  25. Maturity of engineers
    vs maturity of Heroku
    Solution: Operational Complexity
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  26. Simplicity of using
    Apache Kafka in
    Heroku
    Solution: Operational Complexity
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  27. • Start with a monolith and break
    out services from there
    • Use Heroku ¯\_(ツ)_/¯
    • Isolate consistency concerns
    • Ensure each service is self-
    serving

    e.g. own data stores
    • Handle via UI, acknowledgement
    • Use event stream

    e.g. Apache Kafka
    Distribution
    Eventual
    Consistency
    Operational
    Complexity
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  28. How the approach grows
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  29. @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2
    Streaming
    Platform
    Store 1 Store 2 Online Store Distribution Centre Online Marketing
    POS
    Stream
    Inventory
    Stream
    Remarketing
    Stream
    Retail
    Monitoring Reporting
    Data
    Warehouse
    Inventory
    Service
    Logistics
    Service
    Purchasing
    Service
    Profile
    Update
    Service
    Remarketing
    Service
    Source: Confluent

    View full-size slide

  30. @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2
    Source: AWS

    View full-size slide

  31. Elephant in the room:
    Scaling
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    View full-size slide

  32. @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2
    Monoliths can scale more than you might think…
    e.g.

    View full-size slide

  33. take a photo of this slide, or download at https://bit.ly/2zx3TW2
    @ryantownsend / @crc Slides: https://bit.ly/2zx3TW2

    Last year’s talk - more on Event Streams

    https://hrku.co/df17-event-streams

    Microservices Resource Guide

    https://hrku.co/microservices-guide

    Deconstructing Monolithic Applications into Services

    https://hrku.co/deconstructing-monoliths

    How Small Should Your Microservices Be?

    https://hrku.co/microservice-sizing
    Next Steps
    Summary

    Monoliths are okay - microservices aren’t a silver
    bullet

    Start with a monolith - migrate as a journey not a
    single project

    Having a centralised source of truth will set you up
    for success - Apache Kafka is a great solution

    Heroku makes it quick and easy to get started and
    provides the DevOps maturity

    View full-size slide