$30 off During Our Annual Pro Sale. View Details »

AWS Design Patterns at Anki

Ben Whaley
January 21, 2014

AWS Design Patterns at Anki

Application & systems design, deployment, provisioning and tooling for cloud services at Anki. Presented at the AdvancedAWS meetup on Jan 21, 2014 at Anki HQ.

http://www.meetup.com/AdvancedAWS/events/153714572/

Ben Whaley

January 21, 2014
Tweet

More Decks by Ben Whaley

Other Decks in Technology

Transcript

  1. AWS Design Patterns at Anki
    Advanced AWS Meetup
    1/21/2014
    Ben Whaley | @iAmTheWhaley | [email protected]

    View Slide

  2. Work for Anki!
    Anki needs beta testers!

    View Slide

  3. ● Anki and AWS
    ● Application and Systems design
    ● Deployment and Provisioning
    ● Demo
    ● Partnerships
    Overview

    View Slide

  4. View Slide

  5. Services, analytics and infrastructure
    ● Provides web & analytics services for
    Anki
    ● Example services:
    ○ Gameplay diagnostics
    ○ Game replays
    ○ Customer service platform
    ○ Future Anki DRIVE enhancements

    View Slide

  6. Application design

    View Slide

  7. ● Application processes are stateless
    Application design

    View Slide

  8. ● Application processes are stateless
    ● Loose coupling among services
    Application design

    View Slide

  9. ● Application processes are stateless
    ● Loose coupling among services
    ● Configuration is in the environment
    Application design

    View Slide

  10. ● Application processes are stateless
    ● Loose coupling among services
    ● Configuration is in the environment
    ● Built for continuous deployment
    Application design

    View Slide

  11. Systems design

    View Slide

  12. ● n+2 redundancy
    Systems design

    View Slide

  13. Systems design
    ● n+2 redundancy
    ● Compute resources are ephemeral

    View Slide

  14. Systems design
    ● n+2 redundancy
    ● Compute resources are ephemeral
    ● Managed services preferred

    View Slide

  15. Systems design
    ● n+2 redundancy
    ● Compute resources are ephemeral
    ● Managed services preferred
    ● Infrastructure as code

    View Slide

  16. Systems design
    ● n+2 redundancy
    ● Compute resources are ephemeral
    ● Managed services preferred
    ● Infrastructure as code
    ● Single purpose VPCs

    View Slide

  17. Systems design
    ● n+2 redundancy
    ● Compute resources are ephemeral
    ● Managed services preferred
    ● Infrastructure as code
    ● Single purpose VPCs
    ● Contract between backing resources and
    applications

    View Slide

  18. Backing resources
    Public Web Services
    ELBs
    Infrastructure Services
    Bastion, NAT, CM, Metrics
    Private Web Services
    Persistence
    VPC
    External AWS Services
    S3, SQS, SNS
    Inbound requests
    from clients

    View Slide

  19. Deployment and Provisioning

    View Slide

  20. Goal: Continuous deployment
    Source
    Repo
    commit GitHub
    Web Hook
    Deploy
    Worker
    Old Release
    Remains
    Running
    Integration
    Tests
    Deploy to
    Integration
    Deploy to
    Production
    Integration
    tests pass
    SQS
    New
    Release
    Live
    Integration
    tests fail
    Notify
    Check for
    commits
    Commit
    found
    DNS Change

    View Slide

  21. Deployment and Provisioning
    ● New code, new instances

    View Slide

  22. Deployment and Provisioning
    ● New code, new instances
    ● Instance store-backed AMIs built with
    Packer

    View Slide

  23. ● New code, new instances
    ● Instance store-backed AMIs built with
    Packer
    ● AMIs ship with base Docker containers
    Deployment and Provisioning

    View Slide

  24. Deployment and Provisioning
    ● New code, new instances
    ● Instance store-backed AMIs built with
    Packer
    ● AMIs ship with base Docker containers
    ● Instances provisioned with Ansible

    View Slide

  25. Userdata:
    1. Sets configuration variables
    2. Downloads ansible playbooks
    3. Runs ansible
    3-step
    provisioning
    pipeline
    Deploy tool:
    1. Loads configuration from S3
    2. Sets EC2 userdata bash script
    3. Launches instances
    Ansible:
    1. Configures supporting software
    2. Downloads application code
    3. Builds & runs Docker containers
    4. Sends notifications

    View Slide

  26. ● Base container, service container, release
    container
    ● Local dev == Production
    ● Exceptional community
    ● Challenges
    ○ Rapidly changing
    ○ Logging
    ○ Debugging

    View Slide

  27. ● Uses troposphere library to create
    CloudFormation templates
    ● Reusable resource templates
    ● Saves app configuration as json files in S3
    ● Creates CF stacks & kicks off instance
    provisioning
    ● Supports multiple deployments & releases
    Custom deployment tool

    View Slide

  28. ● IAM roles and policies created via
    CloudFormation
    ● Intra-VPC traffic authorized by SG ID
    ● Precise subnetting allows for future network
    ACLs
    ● … some credentials still laying around in
    plain text
    On security

    View Slide

  29. VPC
    Persistence
    Persistence
    Persistence
    RDS
    Multi-AZ
    Persistence
    Persistence
    Private Web Services
    Queue
    Processor
    Query
    Tool
    Persistence
    Persistence
    Public Web Services
    ELB
    Persistence
    Persistence
    Infrastructure Services
    Bastion NAT
    SQS
    Inbound
    Requests
    Flickr Demo App

    View Slide

  30. Deployment Demo

    View Slide

  31. Tooling & design limitations
    ● Blue/Green deployment & ELB pre-warming
    ● “Static” infrastructure templates
    ● Higher latency for requests between VPCs
    ● Ephemeral instance tradeoffs
    ○ Slower to boot than EBS
    ○ Slower to build AMIs
    ● Incomplete functionality (for now)

    View Slide

  32. View Slide

  33. VPC
    Persistence
    Persistence
    Persistence
    Persistence
    Persistence
    Private Web
    Services
    Persistence
    Persistence
    Public Web
    Services
    Persistence
    Persistence
    Collector
    Log collection and analytics

    View Slide

  34. Credits
    ● The 12 Factor App - http://12factor.net
    ● Werner Vogels 2012 re:Invent Keynote
    ● Resty - https://github.com/micha/resty
    ● Troposphere Python cloudformation library
    ○ https://github.com/cloudtools/troposphere/
    ● Continuous deployment at Quora
    ○ http://engineering.quora.com/Continuous-
    Deployment-at-Quora

    View Slide