Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Calling to the future

Eloy Coto
September 17, 2016

Calling to the future

Talk given by Eloy Coto on Pycon-Galicia. September 2016.

Eloy Coto

September 17, 2016

More Decks by Eloy Coto

Other Decks in Programming


  1. ✓ Buzzwords are double-edged sword ✓ Sometimes these tools are

    useful, but not production ready ✓ New tools implied share that knowledge into the company
  2. ✓ Buzzwords are double-edged sword ✓ Sometimes these tools are

    useful, but not production ready ✓ New tools implied share that knowledge into the company ✓ Buzzwords were born to fix problems
  3. Codebase ✓ One codebase tracked in revision control ✓ If

    multiple codebases, it’s not an app. It’s a distributed system $ git init
  4. Dependencies ✓ Explicitly declare and isolate dependencies ✓ Do not

    rely on the implicit existence of any system tools $ echo "click==6.6" > req.txt $ pip install -r req.txt
  5. Config $ docker run -d -e "DB=postgresql://py:[email protected]:6432/py" eloycoto/py $ envdir

    envs/prod/ python manage.py runserver $ vault read secret/DBURL ✓ Config stored in the environment ✓ Config is the specific information required to a host deployment of your codebase ✓ Keep your config outside the app or git.
  6. Backing Services ✓ Treat backing services as attached resources. ✓

    Make no distinction between local and third party services
  7. Build, Release, Run ✓ Strictly separate build & run stages

    ✓ Impossible to make changes at runtime ✓ Build = codebase + dependencies + assets ✓ Release = Build + config ✓ Run = run process against release
  8. Processes ✓ Is stateless ✓ Shares nothing with other processes

    ✓ Session on DB storage ✓ Asset pre-compilation
  9. Port Binding ✓ All services are accessible by one Port

    binding ✓ One app can be backing service of another app using port binding
  10. Concurrency ✓ Scale out via process model ✓ Assign work

    to a process type (web, DB, worker) ✓ Can then handle own multiplexing ✓ Processes don’t daemonize or write PID files
  11. Disposability ✓ Cattle, no pets! ✓ Maximise robustness with fast

    startup and graceful shutdown ✓ Then can be started or stopped at any moment. ✓ Job queues FTW
  12. Dev/Prod Parity Dev = Staging = Prod sqlite ≠ postgres

    = postgres postgres = postgres = postgres ✓ Keep development, staging, and production as similar as possible. ✓ CI & deploy ASAP after writing code ✓ Get devs involved in Operations
  13. Logs Cloudwatch + awslogs ElasticSearch + packetbeat + kibana Grafana

    + Influxdb ✓ Treat logs as event streams ✓ Apps never concern themselves with routing or storage of the output stream ✓ Logs should be indexed and capable to analysis
  14. Admin Processes $ python manage.py migrate ✓ Run admin/management task

    as one-off process in an identical environment ✓ Admin code ships with app code
  15. What is AWS lambda? ✓ A compute service that acts

    on events or requests ✓ Scaling is automatic to event rate ✓ Only pay for what is used ✓ Billed in 100ms chunks
  16. Deploy AWS Lambdas Boto Hard Easy ✓ Lambdas are great,

    but deploy/manage is hard ✓ Amazon does not provide any useful tool for that
  17. ✓ Tool to create and deploy lambdas ✓ Heavy use

    of Cloudformation ✓ Python/Javascript/Golang/Java runtimes ✓ Supported integrations - ApiGateway - Cloudwatch events and cron - Dynamodb and S3 events - Kinesis streams
  18. /settings.yml --- project: pycongalicia default-region: us-east-1 code-bucket: gordon-pycongalicia-39b948b8 apps: -

    gordon.contrib.lambdas - pyday apigateway: helloapi: description: My first API resources: /: methods: [GET, POST] integration: lambda: pyday.pyday /pyday/settings.yml --- lambdas: pyday: code: pyday handler: code.handler runtime: python Gordon design
  19. /pyday/pyday/code.py import json def handler(event, context): data = "Hello World!"

    print(data) print("Received Event: " + json.dumps(event, indent=2)) return data
  20. $ gordon build Loading project resources ✓ apigateway:helloapi Loading installed

    applications contrib_lambdas: ✓ lambdas:version pyday: ✓ lambdas:pyday Building project... 0001_p.json 0002_pr_r.json 0003_r.json
  21. $ gordon apply Applying project... 0001_p.json (cloudformation) CREATE_COMPLETE waiting... |

    0002_pr_r.json (custom) ✓ code/contrib_lambdas_version.zip (de19e97c) ✓ code/pyday_pyday.zip (c6f54de9) 0003_r.json (cloudformation) CREATE_COMPLETE waiting... | Project Outputs: LambdaPydayPyday arn:aws:lambda:us-east-1:105374107590:function:dev-pycongal icia-r-PydayPyday-M79TC8KW3O49:current ApigatewayHelloapi https://qootpywo86.execute-api.us-east-1.amazonaws.com/dev
  22. Success stories ✓ 30K page views for $0.21 Examples ✓