Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Algorithms in the real world (NUS Friday Hacks)

Amy Nguyen
February 22, 2019

Algorithms in the real world (NUS Friday Hacks)

Have you ever had to learn an algorithm and thought, "No way is this ever going to come up in real life"? It turns out many of the topics you're learning in class come up in unexpected ways in the software industry. Come learn how Stripe uses algorithms to build the world's global payments network, setting the new standard in online payments

Amy Nguyen

February 22, 2019
Tweet

More Decks by Amy Nguyen

Other Decks in Technology

Transcript

  1. Algorithms in the Real World
    "When am I ever going to need this?"
    Amy Nguyen
    [email protected]
    Software Engineer, Global Team (APAC)

    View Slide

  2. Who am I?
    ● Class of 2015
    ● Used to think I was bad at math
    ● Failed algorithms in university and had to take it twice
    ● Once had to write breadth-first search for a work project!
    ● Still have a job as an engineer

    View Slide

  3. What is Stripe?
    ● Our mission:
    Increase the GDP of the internet.
    ● How I like to think about it:
    Do whatever it takes to help people start
    businesses online.

    View Slide

  4. View Slide

  5. ● Algorithms appear in unexpected places
    ● Unlike university, you're not alone!
    ● You have to think beyond time and space complexity
    ● Algorithms are easier to understand when you have a concrete
    problem and real-world constraints to think about
    Today's takeaways

    View Slide

  6. Three Examples
    1. Rate limiting the Stripe API
    2. Two practical uses for compiler theory
    3. Reducing human bias in machine learning algorithms

    View Slide

  7. Rate limiting the Stripe API

    View Slide

  8. What is rate limiting?
    ● Limit the number of requests per second

    View Slide

  9. What is rate limiting?
    ● Limit the number of concurrent requests

    View Slide

  10. Why rate limit?

    View Slide

  11. 1. You want to limit the impact of one user's spikes
    Why rate limit?

    View Slide

  12. 1. You want to limit the impact of one user's spikes
    2. You're receiving heavy traffic and want to prioritize critical requests
    Why rate limit?

    View Slide

  13. 1. You want to limit the impact of one user's spikes
    2. You're receiving heavy traffic and want to prioritize critical requests
    3. Your systems might be broken
    Why rate limit?

    View Slide

  14. How do you rate limit?
    1. Pretend you have buckets (one for every user)

    View Slide

  15. How do you rate limit?
    1. Pretend you have buckets (one for every user)
    2. Every N seconds, add a drop to each non-full bucket

    View Slide

  16. How do you rate limit?
    1. Pretend you have buckets (one for every user)
    2. Every N seconds, add a drop to each non-full bucket

    View Slide

  17. How do you rate limit?
    1. Pretend you have buckets (one for every user)
    2. Every N seconds, add a drop to each non-full bucket

    View Slide

  18. How do you rate limit?
    1. Pretend you have buckets (one for every user)
    2. Every N seconds, add a drop to each non-full bucket
    3. For a request, only serve it if you have enough water

    View Slide

  19. Practical uses for compiler theory

    View Slide

  20. Sorbet: A static type checker for Ruby
    ● Uses ASTs, CFGs, and DSLs to parse Ruby code
    ● Abstract Syntax Tree (AST) and Context Free Grammar (CFG):
    Represents the structure of the code and rules for what is valid
    ● Domain-specific Language (DSL):
    Specialized for Ruby type-checking, not a general-purpose language
    ● Written in C++ for performance

    View Slide

  21. Radar: Customized rules for fraud detection

    View Slide

  22. Bias in machine learning decisions

    View Slide

  23. Why did my card get rejected?

    View Slide

  24. Decision trees
    Was the card issued in the US?
    yes
    no
    Were there more than 12
    charges on the card in the last
    30 minutes?
    Were there more than 4
    charges on the card in the last
    30 minutes?
    Fraudulent: 81
    Legitimate: 19
    P(fraud) = 81%
    Fraudulent: 12
    Legitimate: 300
    P(fraud) = 3.8%
    Fraudulent: 12
    Legitimate: 8
    P(fraud) = 60%
    Fraudulent: 4
    Legitimate: 112
    P(fraud) = 3.4%
    no
    yes no yes no

    View Slide

  25. Random forests

    View Slide

  26. View Slide

  27. View Slide

  28. View Slide

  29. Why did my card really get rejected?

    View Slide

  30. "...when you ask a human you’re not really
    getting at the decision process. They make a
    decision first, and then you ask, and then
    they generate an explanation and that may
    not be the true explanation."
    Peter Norvig
    Director of Research, Google

    View Slide

  31. “If I apply for a loan and I get turned down,
    whether it’s by a human or by a machine, and
    I say what’s the explanation, and it says well
    you didn’t have enough collateral. That
    might be the right explanation or it might
    be it didn’t like my skin colour. And I can’t
    tell from that explanation.”
    Peter Norvig
    Director of Research, Google

    View Slide

  32. How could bias appear?

    View Slide

  33. How to measure bias
    Fraud Not fraud
    Flagged Positive False positive
    Not flagged False negative Negative
    ● Rate of true positives (good users are good)
    ● Rate of false negatives (good users aren't rejected disproportionately)

    View Slide

  34. Why can't we just fix it?
    ● Complexity
    ● User experience
    ● Cost
    ● Prioritization
    ● Legality

    View Slide

  35. View Slide

  36. "Haven't I taught you anything? What
    have I always told you? Never trust
    anything that can think for itself if you
    can't see where it keeps its brain?"
    Arthur Weasley

    View Slide

  37. ● Algorithms appear in unexpected places
    ● Unlike university, you're not alone!
    ● You have to think beyond time and space complexity
    ● Algorithms are easier to understand when you have a concrete
    problem and real-world constraints to think about
    Today's takeaways

    View Slide

  38. Thanks!
    ● We're hiring interns and new graduates: stripe.com/jobs
    ● Slides: speakerdeck.com/amyngyn

    View Slide