Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CPU: Container per User

Phil Jay
September 12, 2017

CPU: Container per User

An experiment by Philip Jay & Jun Wang in the ultimate user session isolation. What happens if each user session has it's very own app-server in a container?

See: https://www.youtube.com/watch?v=jH7W8zJtQpg

Phil Jay

September 12, 2017
Tweet

Other Decks in Technology

Transcript

  1. Inspiration (~2009) • Ericsson Erlang • “Dynamic worker” (process) per

    user session • Blast radius of 1 • Resilient to heterogeneous clients
  2. Mobile signalling SGSN / MME / GGSN / SGW …

    Controller Board Session Handler Board 1 Session Handler Board 2 Session Handler Board n . . . . . 1. I’d like to connect to the network 2. Sure. Any future signalling please use Session Handler Board 2, at TEID d33db34f
  3. Mobile signalling SGSN / MME / GGSN / SGW …

    Controller Board Session Handler Board 1 Session Handler Board 2 Session Handler Board n . . . . . 1. I’d like to connect to the network 3. Hey, I’m changing cells 4. ACK 2. Sure. Any future signalling please use Session Handler Board 2, at TEID d33db34f
  4. Upgrades • DO NOT DROP CALLS • Zero-downtime deploys: •

    Existing sessions continue • New sessions start on upgraded software • Similar to ELB connection draining
  5. Isolation & Resilience • Smallest blast radius possible: 1 user

    session • Security: if malicious user,… • Resilience: if memory leaks / bugs,… • Isolation: if poorly implemented client,…
  6. General idea Dispatcher function Session handler function 1. Request session

    2. Start session handler 3. Return session handler details 4. Begin session
  7. “Serverless” (~2016) • AWS Lambda • Docker cluster as a

    service (Joyent, Azure, Openshift,…) • Can we do this with someone else’s servers?
  8. CPU @ REA Hack Days 28 (2017) AWS API Gateway

    Azure Container Instance 1. HTTP GET 3. Docker start 6. HTTP 200 7. HTTP GET / websocket upgrade AWS Lambda 2. Lambda Invoke 4. Public IP:Port 5. Return details
  9. CPU @ REA Hack Days 28 (2017) AWS API Gateway

    Azure Container Instance 1. HTTP GET 3. Docker start 6. HTTP 200 7. HTTP GET / websocket upgrade AWS Lambda 2. Lambda Invoke 4. Public IP:Port 5. Return details
  10. API Gateway+Lambda: Chalice Framework AWS API Gateway AWS Lambda @app.route('/')

    def index(): return {'hello': 'world'} Function decorator Function
  11. CPU @ REA Hack Days 28 (2017) AWS API Gateway

    Azure Container Instance 1. HTTP GET 3. Docker start 6. HTTP 200 7. HTTP GET / websocket upgrade AWS Lambda 2. Lambda Invoke 4. Public IP:Port 5. Return details
  12. Azure Container Instances (beta) • Dropped it after day #1…

    • No Python SDK for beta service (yet..) • Couldn’t get Python in Lambda to subprocess `azure-cli` • Containers don’t delete on exit • In fact, it’s worse: containers restart on exit
  13. AWS ECS to the rescue! ECS Cluster / Autoscale group

    EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 Container Container Container • Easy to get started via AWS Console • Hourly cost for EC2’s • Not “serverless”
  14. AWS ECS to the rescue! Task Definition ECS Cluster /

    Autoscale group EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 Container Container Container Container • Task definition is similar to ASG launch configuration • Specifies containers to launch
  15. Existing application to hack on? • VoIP? • Chat /

    IM? • Something that uses websockets for any reason?
  16. REA Now • Fancy UI showing realtime updates to REA

    listings • NodeJS backend • Listen to AWS Kinesis for data • Stream via a websocket to Javascript client in a browser
  17. REA Now Architecture Javascript in S3 HTTP Websocket Node.JS App

    in Docker on EC2 Listing updates Kinesis stream
  18. REA Now + CPU Architecture Listing updates Javascript in S3

    HTTP GET Lambda invoke ECS start task ∙ rand uuid Return – ∙ Public IP : Port ∙ Random UUID
  19. REA Now + CPU Architecture Listing updates Javascript in S3

    HTTP Websocket Node.JS App in Docker on ECS Cluster Kinesis stream HTTP GET Lambda invoke ECS start task ∙ random uuid ENV: random uuid http://ip:port/uuid Return – ∙ Public IP : Port ∙ Random UUID
  20. REA Now + CPU Architecture Listing updates Javascript in S3

    HTTP Websocket Node.JS App in Docker on ECS Cluster Kinesis stream HTTP GET Lambda invoke ECS start task ∙ rand uuid ENV: random uuid http://ip:port/uuid Return – ∙ Public IP : Port ∙ Random UUID
  21. New & Modified • New: Chalice Application (io28-cpu) for “dispatcher

    function” • Modified: rea-now-feed (“session handler”) – • exit server on disconnect • websocket path => uuid env var • Modified: rea-now-ui (“fancy UI”) – • alter websocket connect, to, API gateway fetch • get websocket client lib from appserver, rather than webpack compiled in
  22. What we learned • It’s something new & different! •

    Initial container pull from repository is slow • Application is slow to start & connect to Kinesis • Chalice is awesome to work with • ECS is easy to use, smooth to transition from docker-in-EC2 style ASGs • The pattern would need more work for production workloads