$30 off During Our Annual Pro Sale. View Details »

Betting on Observability at Simplebet

Betting on Observability at Simplebet

Register and watch this talk now! https://o11yfest.org/attend

Adopting Observability can be an organizational challenge. At Simplebet, we introduced OpenTelemetry and o11y successfully, then published a case study on our experience with Lightstep. We will share our experience gaining traction so that you can learn how to introduce o11y at your company.

Simplebet introduced its first products into the market in 2020, which required a shift in organizational thinking around software deployments, operations, scale, etc. As we geared up for our first public launch, we needed to overcome the challenges of scaling a backend to support thousands of concurrent users at the onset. With the experience of team members who work on the Erlang Observability Working Group, we made a case for introducing distributed tracing with Lightstep as a tool to help us scale. After the first team adopted Otel and made use of distributed tracing, we mechanically implemented the same strategy to other teams in our organization. Through this instrumentation, we were able to isolate performance-critical operations, build evidence for long-standing issues, and change the way the team thought about our production system.

This talk will spend time talking about this experience but spend the majority of the time offering practical guidance for how a developer can gain traction with o11y and Otel at their organization.

Dave Lucia

May 18, 2021
Tweet

Other Decks in Technology

Transcript

  1. Hello!
    I am Dave Luci
    a

    I am a technology leader at Simplebet, active in the
    Elixir community, and new to the o11y movemen
    t

    You can find me at @davydog187
    1

    View Slide

  2. Betting on Observability at

    View Slide

  3. What is Simplebet
    ?

    We turn every moment of a sports
    event into a betting opportunit
    y

    ◇ Odds as a servic
    e

    ◇ Powered by Machine-Learning


    ◇ Craft branded end-user
    experiences
    3

    View Slide

  4. Releasing products
    Our first public product release necessitated
    a change in organizational thinking
    4

    View Slide

  5. How do you build a project that will need to scale up immediately?
    5

    View Slide

  6. Why o11y?
    A successful adoption of o11y starts
    with understanding its value
    6

    View Slide

  7. Adopting o11y
    7
    1 3 5
    6
    4
    2
    Understand your need
    for observability Executive buy-in Instrument your product
    Communicate and
    demonstrate its value to
    your organization
    Choose a vendor Tend your o11y garden

    View Slide

  8. Understanding your needs for
    observability
    O11y has many dimensions. The state of your o11y adoption is not
    binary.
    1

    View Slide

  9. A path towards o11y
    Logs APM Metrics
    9
    Tracing

    View Slide

  10. “ Observability starts with gathering the data at the right level of abstraction, organized around the request path,
    such that you can slice and dice and group and look for patterns and cross-correlations in the requests


    — Charity Majors
    10

    View Slide

  11. Communicate and demonstrate
    o11y’s value to your organization
    2

    View Slide

  12. Benefits of tracing visualization
    MTT
    R

    Reduce the amount of
    time it takes to resolve a
    production issue
    N+1 Querie
    s

    Visually discover
    performance problems
    Measure, then decid
    e

    Squash vague
    arguments like “for
    performance reasons”
    12

    View Slide

  13. Executive buy-in
    Observability isn’t free. Translate organizational value into impact
    3

    View Slide

  14. o11y
    o11y has impact.
    Culture Quality
    14

    View Slide

  15. Choose a vendor
    Some try to do everything, others specializ
    e

    Pick what works for your organization.
    4

    View Slide

  16. 16

    View Slide

  17. Instrument your product
    You need to start somewhere
    5

    View Slide

  18. Where to start
    ◇ Familiarize yourself with Ote
    l

    ◇ Leverage framework instrumentatio
    n

    ◇ Start at the boundarie
    s

    Focus on instrumenting the areas of value and risk to your
    product.
    18

    View Slide

  19. Tend your o11y garden
    Observability needs maintenance to suit your needs, it is evergreen
    6

    View Slide

  20. Grow your
    o11
    y

    Pull weeds,
    fertilize,


    plant.
    20

    View Slide

  21. Engage with the community
    ◇ Follow community leaders on social
    medi
    a

    ◇ Get involved with Ote
    l

    ◇ Be a lurker!
    21

    View Slide

  22. Thanks!
    Any questions
    ?

    You can find me at
    :

    ◇ @davydog18
    7

    [email protected]
    22

    View Slide