Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Beware of Survivorship Bias! (JCON)

Beware of Survivorship Bias! (JCON)

Most talks on a typical conference schedule contain success stories of technology. This could lead to survivorship bias. Survivorship bias causes you to draw false conclusions because you mostly heard about successes, but hardly ever about failures. Yet failures provide us with lots of valuable knowledge: when not to apply a certain technique, pattern or process.

So no success stories in this talk! Just some ‘silver bullets’ we tried to use to solve our problems, but turned out to be Very Bad Ideas™. We’ll share how survivorship bias can easily influence your ideas and cloud your judgement.

After attending this talk you’ll be more aware of survivorship bias and what you can do to keep your head cool, no matter how many ‘silver bullets’ are fired at you.

Maarten Mulders

June 20, 2023
Tweet

More Decks by Maarten Mulders

Other Decks in Technology

Transcript

  1. Beware of
    Survivorship Bias!
    Hanno Embregts
    Maarten Mulders
    @hannotify
    @mthmulders
    #SurvivorshipBias

    View full-size slide

  2. #SurvivorshipBias

    View full-size slide

  3. #SurvivorshipBias

    View full-size slide

  4. https://nos.nl/data/image/2017/01/29/349992/xxl.jpg
    #SurvivorshipBias

    View full-size slide

  5. https://nl.wikipedia.org/wiki/Bestand:Verkiezingen2.jpg
    #SurvivorshipBias

    View full-size slide

  6. ¯\_(

    )_/¯
    #SurvivorshipBias

    View full-size slide

  7. #SurvivorshipBias

    View full-size slide

  8. ¯\_(

    )_/¯
    #SurvivorshipBias

    View full-size slide

  9. https://www.pexels.com/photo/grayscale-photography-of-man-and-woman-sitting-on-chair-1181735
    #SurvivorshipBias

    View full-size slide

  10. Do I need a blockchain?
    No.
    https://medium.com/@sbmeunier/when-do-you-need-blockchain-decision-models-a5c40e7c9ba1
    #SurvivorshipBias

    View full-size slide

  11. Do you need a database?
    Don't use
    a blockchain
    Does it require shared
    write access?
    Are writers known
    and trusted?
    Do you want/need
    to use a trusted 3rd
    party?
    Yes
    Yes
    No
    No
    No
    Are writers'
    interests
    unified?
    Yes
    Yes
    No
    Do you need to
    control functionality?
    No
    Yes
    Do you want transactions
    to be public or private?
    No
    Use a public
    blockchain
    Public
    Where is consensus
    determined?
    Use a hybrid
    blockchain
    Private
    Inter-firm
    Use a private
    blockchain
    Intra-firm
    Yes
    https://medium.com/@sbmeunier/when-do-you-need-blockchain-decision-models-a5c40e7c9ba1
    #SurvivorshipBias

    View full-size slide

  12. Can you articulate a
    real business problem?
    Try a normal
    database.
    Could this have
    been fixed before
    blockchains?
    Is this a 'digital
    identities' and 'blockchains
    are free' play?
    Should or could an entity
    have overall control?
    Yes
    No
    No
    Yes
    Yes
    Will all
    participants need
    to upgrade/replace
    systems?
    No
    Yes
    Will participants mind their
    data being visible?
    No
    You may have
    a use case
    No
    Can they adopt?
    Do the benefits justify the
    costs of adoption?
    Yes
    Yes
    Yes
    https://medium.com/@sbmeunier/when-do-you-need-blockchain-decision-models-a5c40e7c9ba1
    #SurvivorshipBias

    View full-size slide

  13. election blockchain
    writers known?
    yes no
    writers trusted?
    yes no
    writers' interests uni ed?
    yes no
    trusted third party?
    yes no
    private transactions?
    yes no
    #SurvivorshipBias

    View full-size slide

  14. Survivorship
    Survivorship
    Survivorship
    Survivorship
    Survivorship
    Bias
    Bias
    Bias
    Bias
    Bias
    https://pxhere.com/nl/photo/973302
    #SurvivorshipBias

    View full-size slide

  15. https://commons.wikimedia.org/wiki/File:Survivorship-bias.png
    #SurvivorshipBias

    View full-size slide

  16. It won't solve
    It won't solve
    It won't solve
    It won't solve
    It won't solve
    everything!
    everything!
    everything!
    everything!
    everything!
    Just because a conference speaker solved
    his/her problem with some new, cool
    technique doesn't mean it will also solve
    yours.
    https://www.pexels.com/photo/2-hands-holding-1-jigsaw-puzzle-piece-each-164531/
    #SurvivorshipBias

    View full-size slide

  17. Conference-driven
    Conference-driven
    Conference-driven
    Conference-driven
    Conference-driven
    development
    development
    development
    development
    development
    Learn new skills at a conference, and then
    use it to solve pretty much everything.
    https://pxhere.com/en/photo/1130503
    #SurvivorshipBias

    View full-size slide

  18. Overengineering
    Overengineering
    Overengineering
    Overengineering
    Overengineering
    If you apply CDD to everything you do,
    your solutions will quickly become
    overengineered.
    https://www. ickr.com/photos/footfun/2454000816
    #SurvivorshipBias

    View full-size slide

  19. http://gph.is/18wDuA0
    #SurvivorshipBias

    View full-size slide

  20. https://en.wikipedia.org/wiki/
    List_of_failed_and_overbudget_custom_software_projects
    #SurvivorshipBias

    View full-size slide

  21. #SurvivorshipBias

    View full-size slide

  22. https://github.com/MinBZK/OperatieBRP
    https://github.com/MinBZK/OperatieBRP
    https://github.com/MinBZK/OperatieBRP
    https://github.com/MinBZK/OperatieBRP
    https://github.com/MinBZK/OperatieBRP
    #SurvivorshipBias

    View full-size slide

  23. #SurvivorshipBias

    View full-size slide

  24. #SurvivorshipBias

    View full-size slide

  25. https://pxhere.com/en/photo/655299
    #SurvivorshipBias

    View full-size slide

  26. Alternatives
    Alternatives
    Alternatives
    Alternatives
    Alternatives
    https://pxhere.com/en/photo/1267965
    #SurvivorshipBias

    View full-size slide

  27. Beating
    Beating
    Beating
    Beating
    Beating
    Survivorship
    Survivorship
    Survivorship
    Survivorship
    Survivorship
    Bias
    Bias
    Bias
    Bias
    Bias
    https://pxhere.com/en/photo/958105
    #SurvivorshipBias

    View full-size slide

  28. Bullet image by , from
    Clker-Free-Vector-Images Pixabay
    #SurvivorshipBias

    View full-size slide

  29. Understand the
    Understand the
    Understand the
    Understand the
    Understand the
    problem first
    problem first
    problem first
    problem first
    problem first
    https://pxhere.com/en/photo/843810
    #SurvivorshipBias

    View full-size slide

  30. Track A Track B
    09:00-
    09:50
    QWERTY or DVORAK?
    Debunking the Keyboard
    Layout Myths
    Beware of
    Survivorship Bias!
    10:00-
    10:50
    How I Created a MMORPG
    Game Using Haskell Only
    An Introduction to
    Kubernetes
    11:00-
    11:50
    Monitoring Kubernetes with
    Prometheus
    Return of the
    Blockchain
    #SurvivorshipBias

    View full-size slide

  31. Gain some experience
    Gain some experience
    Gain some experience
    Gain some experience
    Gain some experience
    first
    first
    first
    first
    first
    This could mean you have to wait until
    next year's conference in order to
    appreciate fully the contents of an
    advanced talk.
    https://pxhere.com/en/photo/918274
    #SurvivorshipBias

    View full-size slide

  32. Build your personal
    Build your personal
    Build your personal
    Build your personal
    Build your personal
    conference schedule
    conference schedule
    conference schedule
    conference schedule
    conference schedule
    accordingly
    accordingly
    accordingly
    accordingly
    accordingly
    Value topics relevant to your experience
    over 'the next silver bullet'.
    https://pxhere.com/en/photo/1370392
    #SurvivorshipBias

    View full-size slide

  33. Make sure you meet
    Make sure you meet
    Make sure you meet
    Make sure you meet
    Make sure you meet
    the prerequisites
    the prerequisites
    the prerequisites
    the prerequisites
    the prerequisites
    Don't attend a talk that proposes a
    solution to a problem, if you don't
    understand the problem yet.
    https://pxhere.com/en/photo/843810
    #SurvivorshipBias

    View full-size slide

  34. Remember the
    Remember the
    Remember the
    Remember the
    Remember the
    Hype Cycle
    Hype Cycle
    Hype Cycle
    Hype Cycle
    Hype Cycle
    https://pxhere.com/en/photo/1262347
    #SurvivorshipBias

    View full-size slide

  35. https://upload.wikimedia.org/wikipedia/commons/b/bf/Hype-Cycle-General.png
    #SurvivorshipBias

    View full-size slide

  36. Possible Talk Title
    On the Rise NoSQL: The New Reality
    At the Peak NoSQL Will Change Your Life
    and Fix Your Projects
    Sliding Into the
    Trough
    How We Do NoSQL at Cool, Inc.
    Climbing the
    Slope
    Modern NoSQL Patterns and
    Practices
    Entering the
    Plateau
    NoSQL 2.0 with SilverBulletDB
    #SurvivorshipBias

    View full-size slide

  37. Does the speaker
    Does the speaker
    Does the speaker
    Does the speaker
    Does the speaker
    mention any
    mention any
    mention any
    mention any
    mention any
    drawbacks?
    drawbacks?
    drawbacks?
    drawbacks?
    drawbacks?
    The best talks contain detailed
    comparisons of pros
    and cons
    https://upload.wikimedia.org/wikipedia/commons/b/b2/Balance_scale_IMGP9728.jpg
    #SurvivorshipBias

    View full-size slide

  38. Use the Q&A at the
    Use the Q&A at the
    Use the Q&A at the
    Use the Q&A at the
    Use the Q&A at the
    end of the talk
    end of the talk
    end of the talk
    end of the talk
    end of the talk
    Scary at rst perhaps, but take comfort in
    the fact that the rest of the audience is
    rooting for you.
    https://www.pexels.com/photo/monopoly-car-piece-1634213/
    #SurvivorshipBias

    View full-size slide

  39. Try to arrange a
    Try to arrange a
    Try to arrange a
    Try to arrange a
    Try to arrange a
    'hallway meeting'
    'hallway meeting'
    'hallway meeting'
    'hallway meeting'
    'hallway meeting'
    Which basically means just bumping into
    them in the hallways!
    https://pxhere.com/en/photo/1454179
    #SurvivorshipBias

    View full-size slide

  40. They seem to really like that, by the way!
    #SurvivorshipBias

    View full-size slide

  41. How would you
    you
    you
    you
    you solve
    the puzzle?
    https://pxhere.com/en/photo/1207943
    #SurvivorshipBias

    View full-size slide

  42. Start tinkering
    Start tinkering
    Start tinkering
    Start tinkering
    Start tinkering
    https://pxhere.com/en/photo/901709
    Learning Through Tinkering (Tom Cools)
    #SurvivorshipBias

    View full-size slide

  43. Experiment
    Experiment
    Experiment
    Experiment
    Experiment
    If you cannot reproduce the problem
    and
    the solution, there must be another factor
    at play.
    https://pxhere.com/en/photo/1004202
    #SurvivorshipBias

    View full-size slide

  44. Look at the numbers
    Look at the numbers
    Look at the numbers
    Look at the numbers
    Look at the numbers
    Succeeded Failed
    Same approach
    Di erent approach

    What do they tell you?
    https://pxhere.com/en/photo/1437973
    #SurvivorshipBias

    View full-size slide

  45. The community needs
    The community needs
    The community needs
    The community needs
    The community needs
    you!
    you!
    you!
    you!
    you!
    Blog, tweet or speak about your failures
    and what you've learned from them!
    #SurvivorshipBias

    View full-size slide

  46. Search StackOverflow
    Search StackOverflow
    Search StackOverflow
    Search StackOverflow
    Search StackOverflow
    Blogs and videos will tell you the happy
    path. StackOver ow will tell you the non-
    happy path.
    https://pxhere.com/en/photo/625549
    #SurvivorshipBias

    View full-size slide

  47. Master Your Tools
    Master Your Tools
    Master Your Tools
    Master Your Tools
    Master Your Tools
    Use your tools long enough to
    really get to
    know them.
    https://pxhere.com/en/photo/704019
    #SurvivorshipBias

    View full-size slide

  48. Maven's Dependency Tree
    mvn dependency:tree | grep spring-batch
    mvn dependency:tree | grep spring-batch -C 3
    mvn dependency:tree -Dincludes=*:spring-batch*:*
    #SurvivorshipBias

    View full-size slide

  49. Tempting
    Tempting
    Tempting
    Tempting
    Tempting
    Abraham Maslow
    Toward a Psychology of Being
    "I suppose it is tempting, if the only tool
    you have is a hammer, to treat everything
    as if it were a nail."
    https://www.pexels.com/photo/black-claw-hammer-on-brown-wooden-plank-209235/
    #SurvivorshipBias

    View full-size slide

  50. Take Aways
    Take Aways
    Take Aways
    Take Aways
    Take Aways
    1. Share your failures and lessons learned!
    2. Start tinkering!
    3. See you in the Hallway track!
    #SurvivorshipBias

    View full-size slide

  51. Thank you!
    bit.ly/survivorship-bias-jcon
    hanno.codes maarten.mulders.it
    @hannotify @mthmulders
    #SurvivorshipBias

    View full-size slide