$30 off During Our Annual Pro Sale. View Details »

Scale Oriented Architecture

Scale Oriented Architecture

This talk looks at the science of building web services, thinking about APIs from their most basic primitives, and the lessons learned at Twilio and Runscope for supporting APIs at massive scale. (Presented at Silicon Valley Code Camp 2013)

Frank Stratton ☺

October 05, 2013
Tweet

More Decks by Frank Stratton ☺

Other Decks in Programming

Transcript

  1. Scale-Oriented
    Architecture
    Frank Stratton
    @FrankRStratton

    View Slide

  2. Runscope

    View Slide

  3. Runscope
    API
    Client
    HTTP
    API
    Server
    HTTP

    View Slide

  4. Runscope

    View Slide

  5. Runscope

    View Slide

  6. Runscope
    API
    Client
    HTTP
    API
    Server
    HTTP

    View Slide

  7. View Slide

  8. View Slide

  9. Dashboard
    Traffic
    Gateway
    API

    View Slide

  10. Dashboard
    Traffic
    Gateway
    API
    Request Vault Identity

    View Slide

  11. Dashboard
    Traffic
    Gateway
    API
    Request Vault Identity
    Eventador
    Calculon
    Courier
    Warp Pipe
    Mission
    Control
    Prometheus
    Vivaldi
    Passageway

    View Slide

  12. Dashboard
    Traffic
    Gateway
    API
    Request Vault Identity
    Eventador
    Calculon
    Courier
    Warp Pipe
    Mission
    Control
    Prometheus
    Vivaldi
    Passageway

    View Slide

  13. “The largest architectural change we
    made was to move from our monolithic
    Ruby application to one that is more
    services oriented.”
    https://blog.twitter.com/2013/new-tweets-per-second-record-and-how

    View Slide

  14. View Slide

  15. S
    O
    A
    ERVICE
    RIENTED
    RCHITECTURE
    S

    View Slide

  16. S
    O
    A
    RIENTED
    RCHITECTURE
    S

    View Slide

  17. S
    O
    A
    RIENTED
    RCHITECTURE
    CALE
    S

    View Slide

  18. SOA 2.0

    View Slide

  19. SOA 2.0
    Scalable Architectures

    View Slide

  20. SOA 2.0
    Scalable Architectures
    Scalable Teams

    View Slide

  21. Scalable Architectures
    Dashboard
    Traffic
    Gateway
    API
    Request Vault Identity
    Dashboard
    Dashboard
    Dashboard
    Dashboard
    Identity
    Identity
    Identity
    Identity
    Identity
    Request Vault
    Request Vault
    Traffic
    Gateway
    Scale Horizontally
    API
    API

    View Slide

  22. Scalable Architectures
    Identity Identity Identity Identity
    Scale Horizontally
    HaProxy
    Nginx
    etc...

    View Slide

  23. Scalable Architectures
    Dashboard
    Traffic
    Gateway
    API
    Request Vault Identity
    Dashboard
    Dashboard
    Dashboard
    Dashboard
    Identity
    Identity
    Identity
    Identity
    Identity
    Request Vault
    Request Vault
    Traffic
    Gateway
    Scale Horizontally
    API
    API

    View Slide

  24. Scalable Architectures
    Dashboard
    Traffic
    Gateway
    Request Vault Identity
    Dashboard
    Dashboard
    Dashboard
    Dashboard
    Identity
    Identity
    Identity
    Identity
    Identity
    Request Vault
    Request Vault
    Traffic
    Gateway
    Fail Independently
    API
    API
    API

    View Slide

  25. Scalable Architectures
    Fail Independently
    Request Vault Identity
    Eventador
    Calculon
    Courier
    Warp Pipe
    Mission
    Control
    Prometheus
    Vivaldi
    Passageway
    Dashboard
    Traffic
    Gateway
    API

    View Slide

  26. master

    View Slide

  27. master

    View Slide

  28. master

    View Slide

  29. master

    View Slide

  30. master

    View Slide

  31. master

    View Slide

  32. master

    View Slide

  33. master

    View Slide

  34. master

    View Slide

  35. master

    View Slide

  36. master

    View Slide

  37. Scalable Teams
    Request Vault Identity
    Dashboard
    Traffic
    Gateway
    API

    View Slide

  38. Scalable Teams
    Deploy Independently
    Request Vault Identity
    Dashboard
    Traffic
    Gateway
    API

    View Slide

  39. Scalable Teams
    Deploy Independently
    Request Vault Identity
    Dashboard
    Traffic
    Gateway
    API
    v3 v5
    v20
    v45 v2

    View Slide

  40. Dashboard
    Traffic
    Gateway
    API
    Request Vault Identity
    Eventador
    Calculon
    Courier
    Warp Pipe
    Mission
    Control
    Prometheus
    Vivaldi
    Passageway
    Scalable Teams

    View Slide

  41. Dashboard
    Traffic
    Gateway
    API
    Postgres

    View Slide

  42. Identity
    Postgres
    Memcache
    Flask
    Scalable Teams
    Dashboard
    Traffic
    Gateway
    API
    Request Vault

    View Slide

  43. Identity
    GET /teams/
    PUT /teams/
    GET /buckets/
    GET /buckets
    ...
    Scalable Teams
    Dashboard
    Traffic
    Gateway
    API
    Request Vault

    View Slide

  44. GET /teams/
    PUT /teams/
    GET /buckets/
    GET /buckets
    ...

    View Slide

  45. have you built an
    HTTP/REST API?

    View Slide

  46. GET

    View Slide

  47. POST

    View Slide

  48. PUT

    View Slide

  49. PUT
    is idempotent

    View Slide

  50. PUT
    import requests
    resp = requests.put(“http://127.0.0.1/buckets/1234”, data={ ... })
    if resp.ok:
    return resp.json()
    # else, raise an error?!
    raise Exception(“Bucket didn’t return 200 OK”)

    View Slide

  51. PUT
    import requests
    retries = 3
    for i in range(retries):
    resp = requests.put(“http://127.0.0.1/buckets/1234”, data={ ... })
    if resp.ok:
    return resp.json()

    View Slide

  52. Dashboard
    Traffic
    Gateway
    API
    Request Vault Identity
    PUT /requests/
    PUT /responses/

    View Slide

  53. Fail Fast
    & Retry

    View Slide

  54. Dashboard
    Traffic
    Gateway
    API
    Request Vault Request Vault Request Vault

    View Slide

  55. Queue All The
    Things

    View Slide

  56. Redis as a Queue
    import redis
    r = redis.StrictRedis()
    def queue_item(item):
    r.lpush(“work_queue”, item)
    def get_item():
    return r.rpop(“work_queue”)
    class Worker(Greenlet):
    def _run():
    while True:
    item = get_item()
    if item:
    # do work
    else:
    sleep(1)

    View Slide

  57. Redis as a Queue
    import redis
    r = redis.StrictRedis()
    def queue_item(item):
    r.lpush(“work_queue”, item)
    def get_item():
    return r.rpoplpush(
    “work_queue”, “in_progress”)
    def finish_task(item):
    r.lrem(“in_progress”, 1, item)
    class Worker(Greenlet):
    def _run():
    while True:
    item = get_item()
    if item:
    # do work, then
    finish_task(item)
    else:
    sleep(1)

    View Slide

  58. Redis as a Queue
    import redis
    r = redis.StrictRedis()
    def queue_item(item):
    r.lpush(“work_queue”, item)
    def get_item():
    return r.rpoplpush(
    “work_queue”, “in_progress”)
    def finish_task(item):
    r.lrem(“in_progress”, 1, item)
    def cleanup():
    items = r.lrange(
    “in_progress”, 0, -1)
    r.rpush(“work_queue”, *items)
    r.delete(“in_progress”)

    View Slide

  59. Redis as a Queue
    class Worker(Greenlet):
    def _run():
    while True:
    item = get_item()
    if item:
    do_work(item)
    finish_task(item)
    else:
    sleep(1)
    def do_work(item):
    item[‘formatted_name’] = string.capwords(item[‘name’])
    requests.put(“http://my-service/items/item[‘id’]”,
    data=item)

    View Slide

  60. Dashboard
    Traffic
    Gateway
    API
    Request Vault Identity
    Redis

    View Slide

  61. View Slide

  62. Dashboard
    Traffic
    Gateway
    API
    Request Vault Identity
    Eventador
    Calculon
    Courier
    Warp Pipe
    Mission
    Control
    Prometheus
    Vivaldi
    Passageway

    View Slide

  63. See Your Data

    View Slide

  64. Dashboard
    Traffic
    Gateway
    API
    Request Vault Identity
    Eventador
    Calculon
    Courier
    Warp Pipe
    Mission
    Control
    Prometheus
    Vivaldi
    Passageway

    View Slide

  65. Runscope

    View Slide

  66. Questions?
    Frank Stratton
    Twitter: @FrankRStratton

    View Slide