Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deploy Python apps in 5 minutes with a PaaS

Deploy Python apps in 5 minutes with a PaaS

How can you avoid servers and get back to coding? Platform-as-a-service (PaaS) makes deployment easy. But which PaaS do you choose and how do you get started? This talk will examine several of the leading PaaS providers and discuss their pros/cons. We'll also give examples for how to deploy the same app to each of them to see the differences.

PyCon 2013

March 15, 2013
Tweet

More Decks by PyCon 2013

Other Decks in Programming

Transcript

  1. DEPLOY PYTHON APPS IN 5 MINUTES WITH A PAAS Nate

    Aune (@natea) PyCon Santa Clara March 15, 2013 1 Sunday, March 17, 13
  2. 2010-2011 • Nothing equivalent to Heroku for Django developers •

    Built my own PaaS (“How hard can it be?”) • Shut down last year. Increased competition from big corps meant a race to the bottom 3 Sunday, March 17, 13
  3. AGENDA •Why should I care? •What is a PaaS? What

    are the advantages? •Which PaaS should I use? •When might I not want to use a PaaS? 5 Sunday, March 17, 13
  4. WHERE ARE YOU DEPLOYING/HOSTING TODAY? •Shared hosting (i.e. Webfaction) •Running

    your own servers (co-located)? •Using an IaaS provider (i.e. AWS or Rackspace) •Already using a platform-as-a-service (PaaS) 6 Sunday, March 17, 13
  5. CONFIGURING A SERVER •deciding what size to get (memory, disk)

    •getting all the dependencies installed on it •SSHing into the machine to deploy stuff (feels dirty, but he’s under time pressure) 1  day 1  day 9 Sunday, March 17, 13
  6. SCALING 1  day 1  day 1  day 1  day 1

     day 12 Sunday, March 17, 13
  7. te sed 6 Sysadmin $80k DBA $90k Train new guy

    8 days 14 Sunday, March 17, 13
  8. WHAT IS A PAAS? Platform-as-a-Service enables developers to create innovative

    applications without operational overhead around configuration, deployment and management. 17 Sunday, March 17, 13
  9. LAYERS OF INFRASTRUCTURE Source: EngineYard “PaaS - State of the

    Market Survey” May 2012 - http://venturebeat.com/2012/07/05/engine-yard-paas-infographic/ 18 Sunday, March 17, 13
  10. Source: EngineYard “PaaS - State of the Market Survey” May

    2012 - http://venturebeat.com/2012/07/05/engine-yard-paas-infographic/ 19 Sunday, March 17, 13
  11. Source: EngineYard “PaaS - State of the Market Survey” May

    2012 - http://venturebeat.com/2012/07/05/engine-yard-paas-infographic/ 20 Sunday, March 17, 13
  12. Source: EngineYard “PaaS - State of the Market Survey” May

    2012 - http://venturebeat.com/2012/07/05/engine-yard-paas-infographic/ 21 Sunday, March 17, 13
  13. Source: AppFog “Evolution of the Cloud: Toward a NoOps World”

    Jan 2012 http://gigaom.com/cloud/why-2013-is-the-year-of-noops-for-programmers-infographic/ 22 Sunday, March 17, 13
  14. EFFICIENT, ELASTIC, SECURE • Lots of applications co-located on a

    few servers • Drastically reduces resources • Add/remove capacity depending on load • All secured using SELinux or LXC 23 Sunday, March 17, 13
  15. LET THEM DO THE BORING STUFF • Patches and updates

    • Migrating applications • Backups / snapshots • Configuring everything (web servers, load balancers, modules, databases) 24 Sunday, March 17, 13
  16. FOLLOW ALONG! $ git clone http://github.com/appsembler/paasbakeoff $ git branch dotcloud

    elasticbeanstalk gae heroku * master mezzpycon openshift stackato 25 Sunday, March 17, 13
  17. DEFINE OS DEPENDENCIES www: type: python systempackages: - openoffice.org -

    mysql-client-5.1 5  min 27 Sunday, March 17, 13
  18. WHO ARE THE PLAYERS? • CloudFoundry (open source by VMWare)

    • AppFog • Stackato (ActiveState) •Heroku (now Salesforce) •Dotcloud •Gondor (Python only) •OpenShift (Redhat’s open source PaaS) •App Engine (Google) •Elastic Beanstalk (Amazon) •Azure (Microsoft) 34 Sunday, March 17, 13
  19. WHO ARE THE PLAYERS? • CloudFoundry (open source by VMWare)

    • AppFog • Stackato (ActiveState) •Heroku (now Salesforce) •Dotcloud •Gondor (Python only) •OpenShift (Redhat’s open source PaaS) •App Engine (Google) •Elastic Beanstalk (Amazon) •Azure (Microsoft) We’ll look at these ones today. 35 Sunday, March 17, 13
  20. ANATOMY OF A DJANGO APP ON HEROKU Procfile to configure

    process requirements.txt to define dependencies customized settings.py for Heroku if not using S3, must configure static serve 39 Sunday, March 17, 13
  21. INSTALL THE HEROKU TOOLBELT Download the Heroku Toolbelt http://toolbelt.herokuapp.com/osx/download $

    heroku login Enter your Heroku credentials. Email: [email protected] Password: ****** Could not find an existing public key. Would you like to generate one? [Yn] Generating new SSH public key. Uploading ssh public key /Users/someone/.ssh/id_rsa.pub 40 Sunday, March 17, 13
  22. CREATE APP AND DB $ heroku create paasbakeoff Creating paasbakeoff...

    done, stack is cedar http://paasbakeoff.herokuapp.com/ | [email protected]:paasbakeoff.git Git remote heroku added $ heroku addons:add heroku-postgresql:dev Adding heroku-postgresql:dev on paasbakeoff... done, v3 (free) Attached as HEROKU_POSTGRESQL_GREEN_URL Database has been created and is available Use `heroku addons:docs heroku-postgresql:dev` to view documentation. $ heroku pg:info === HEROKU_POSTGRESQL_GREEN_URL Plan: Dev Status: available 41 Sunday, March 17, 13
  23. SET DB ENV VARIABLES $ heroku config === paasbakeoff Config

    Vars HEROKU_POSTGRESQL_GREEN_URL: postgres://x:[email protected]:5432/d2b3c9ichbauv0 $ heroku pg:promote HEROKU_POSTGRESQL_GREEN Promoting HEROKU_POSTGRESQL_GREEN_URL to DATABASE_URL... done $ heroku config === paasbakeoff Config Vars DATABASE_URL: postgres://x:[email protected]:5432/d2b3c9ichbauv0 42 Sunday, March 17, 13
  24. ADD TO SETTINGS.PY $ heroku config:set RACK_ENV=production Set the RACK_ENV

    environment variable to production 43 Sunday, March 17, 13
  25. USE S3 FOR SERVING STATIC AND MEDIA FILES And Sendgrid

    for sending emails 44 Sunday, March 17, 13
  26. PROCFILE web: gunicorn_django -b 0.0.0.0:\$PORT -w 9 -k gevent --max-requests

    250 \ --preload mywebsite/settings.py Configure AWS settings $ heroku config:set AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxxx $ heroku config:set AWS_SECRET_ACCESS_KEY=yyyyyyyyyyyyyyyyyyyyyyyyy $ heroku config:set AWS_STORAGE_BUCKET_NAME=zzzzzzzzzzzzz Sync, migrate, collectstatic $ heroku run python mywebsite/manage.py createdb $ heroku run python mywebsite/manage.py collectstatic 45 Sunday, March 17, 13
  27. DEPLOY WITH GIT PUSH $ git push heroku master Counting

    objects: 8, done. Delta compression using up to 2 threads. Compressing objects: 100% (6/6), done. Writing objects: 100% (6/6), 616 bytes, done. Total 6 (delta 3), reused 0 (delta 0) -----> Heroku receiving push -----> Python/Django app detected -----> Preparing Python interpreter (2.7.2) -----> Creating Virtualenv version 1.7.2 New python executable in .heroku/venv/bin/python2.7 ... Running virtualenv with interpreter /usr/local/bin/python2.7 -----> Activating virtualenv -----> Installing dependencies using pip version 1.1 Downloading/unpacking Django==1.4.2 (from -r requirements.txt (line 1)) ... Cleaning up... -----> Collecting static files 46 Sunday, March 17, 13
  28. ANATOMY OF A DJANGO APP ON HEROKU Procfile to configure

    process requirements.txt to define dependencies customized settings.py for Heroku 47 Sunday, March 17, 13
  29. MORE HEROKU RESOURCES • Getting Started with Django on Heroku

    https://devcenter.heroku.com/articles/django • Heroku Hackers Guide ($10 eBook) http://theherokuhackersguide.com • Developers Guide to Running Django Applications on Heroku http://kencochrane.net/blog/2011/11/developers-guide-for-running-django-apps-on-heroku/ • django-skel http://django-skel.readthedocs.org 48 Sunday, March 17, 13
  30. ANATOMY OF A DJANGO APP ON DOTCLOUD dotcloud.yml to store

    config info nginx.conf to config URL rewriting wsgi.py to serve using uWSGI createdb.py to create the database mkadmin.py to make the admin user postinstall to run syncdb, collectstatic customized settings.py for Dotcloud 52 Sunday, March 17, 13
  31. INSTALL THE DOTCLOUD CLIENT $ sudo pip install dotcloud $

    dotcloud setup dotCloud username or email: [email protected] Password: ==> dotCloud authentication is complete! You are recommended to run `dotcloud check` now. 53 Sunday, March 17, 13
  32. CREATE AND PUSH AN APP $ dotcloud create mezz ==>

    Creating a sandbox application named "mezz" ==> Application "mezz" created. Connect the current directory to "mezz"? [Y/n]: ==> Connecting with the application "mezz" ==> Connected with default push options: --rsync $ dotcloud push ==> Pushing code with rsync from "./" to application mezz building file list ... done 14:17:51: [www.0] Migrating stateful data located in ~/data 14:18:05: [www.0] Launching... 14:18:07: [www.0] Re-routing traffic to the new build... 14:18:08: [www.0] Successfully installed build revision rsync instance #0 54 Sunday, March 17, 13
  33. ANATOMY OF A DJANGO APP ON DOTCLOUD dotcloud.yml to store

    config info nginx.conf to config URL rewriting wsgi.py to serve using uWSGI createdb.py to create the database mkadmin.py to make the admin user postinstall to run syncdb, collectstatic customized settings.py for Dotcloud 55 Sunday, March 17, 13
  34. MORE RESOURCES • Dotcloud Python docs http://docs.dotcloud.com/0.9/services/python/ • Dotcloud Django

    docs http://docs.dotcloud.com/0.9/tutorials/python/django/ • django-on-dotcloud https://github.com/dotcloud/django-on-dotcloud/ • python-on-dotcloud https://github.com/kencochrane/python-on-dotcloud 62 Sunday, March 17, 13
  35. STACKATO by ActiveState Python 3, Run anywhere, New Relic integration

    http://appsembler.com/blog/django-deployment-using-stackato/ 63 Sunday, March 17, 13
  36. ANATOMY OF DJANGO APP ON STACKATO stackato.yml to define services

    customized settings file wsgi.py to serve using uWSGI 66 Sunday, March 17, 13
  37. STACKATO FOR DJANGO 1.Download the Stackato client http://www.activestate.com/stackato/download_client 2.Create a

    wsgi.py file 3.Create a requirements.txt file (if you don’t already have one) 4.Edit the DATABASES and MEDIA_ROOT settings in settings.py file 5.Create a stackato.yml file to persist the 72 Sunday, March 17, 13
  38. TARGET AND LOGIN $ stackato target api.appsembler.net $ stackato login

    --email [email protected] Attempting to login to [https://api.appsembler.net] Password: ******** Successfully logged into [https://api.appsembler.net] 77 Sunday, March 17, 13
  39. INITIAL PUSH $ stackato push Would you like to deploy

    from the current directory ? [Yn]: Would you like to use 'paasbakeoff' as application name ? [Yn]: Detected a Python Application, is this correct ? [Yn]: Framework: python Runtime: <framework-specific default> Application Deployed URL [paasbakeoff.appsembler.net]: Application Url: paasbakeoff.appsembler.net Enter Memory Reservation [128M]: 78 Sunday, March 17, 13
  40. BIND SERVICES What kind of service ? 1. filesystem 2.

    memcached 3. mongodb 4. mysql 5. postgresql 6. rabbitmq 7. redis Choose: 5 Specify the name of the service [postgresql-cf691]: Creating Service: OK Binding Service: OK Create another ? [yN]: Would you like to save this configuration? [yN]: y Uploading Application [paasbakeoff]: 79 Sunday, March 17, 13
  41. AUTO-GENERATED STACKATO.YML FILE name: paasbakeoff instances: 1 framework: type: python

    mem: 128 requirements: pypm: - pillow - psycopg2 Let’s add some other requirements to be installed requirements: pypm: - pillow pip: - psycopg2 -OR- 80 Sunday, March 17, 13
  42. RUNNING SYNCDB $ stackato update -n $ stackato start $

    stackato logs $ stackato run python mywebsite/manage.py syncdb --noinput Creating tables ... ... Creating table django_comments Creating table django_comment_flags Creating default account (username: admin / password: default) ... Creating default Site 127.0.0.1:8000 ... 81 Sunday, March 17, 13
  43. HANDLING STATIC ASSETS $ stackato run python mywebsite/manage.py collectstatic --noinput

    ... Copying '/app/python/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/rtl.css' Copying '/app/python/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/ie.css' Copying '/app/python/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/forms.css' framework: type: python home-dir: app processes: web: $STACKATO_UWSGI --static-map /static=$HOME/mywebsite/static Make sure they were copied to the right place 82 Sunday, March 17, 13
  44. RUN MANAGEMENT COMMANDS hooks: post-staging: - python mywebsite/manage.py syncdb --noinput

    - python mywebsite/manage.py collectstatic --noinput - python mywebsite/manage.py migrate --noinput requirements: pypm: - pillow - psycopg2 - south Make sure you add South to the requirements 83 Sunday, March 17, 13
  45. PERSISTED FILESYSTEM FOR FILE UPLOADS services: postgresql-cf691: postgresql filesystem-paasbakeoff: filesystem

    MEDIA_ROOT = os.environ['STACKATO_FILESYSTEM'] Remember to set the MEDIA_ROOT in settings.py: 84 Sunday, March 17, 13
  46. setup.py instead of requirements.txt application inside of wsgi dir .htaccess

    to serve up static files /data/ dir to store uploaded media files Action hooks for running commands during build, deploy, post-deploy, etc. ANATOMY OF AN OPENSHIFT REPO 89 Sunday, March 17, 13
  47. INSTALL THE RHC CLIENT $ sudo gem install rhc $

    rhc setup ... $ rhc domain status 7 tests, 12 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications 100% passed $ ssh-add ~/.ssh/id_rsa $ ssh-agent You may need to add the SSH key and start the SSH agent 90 Sunday, March 17, 13
  48. QUICK ‘N DIRTY INSTRUCTIONS $ rhc app create -a mezzanineopenshift

    -t python 2.6 $ rhc cartridge add -c mysql-5.1 -a mezzanineopenshift $ cd mezzanineopenshift $ git remote add paasbakeoff git://github.com/appsembler/paasbakeoff.git $ git fetch paasbakeoff $ git merge paasbakeoff/openshift 91 Sunday, March 17, 13
  49. setup.py instead of requirements.txt application inside of wsgi dir .htaccess

    to serve up static files /data/ dir to store uploaded media files Action hooks for running commands during build, deploy, post-deploy, etc. ANATOMY OF AN OPENSHIFT REPO 92 Sunday, March 17, 13
  50. CREATE AND BIND THE DATABASE $ rhc cartridge add -c

    mysql-5.1 -a mezz Password: ****** Adding 'mysql-5.1' to application 'mezz' Success mysql-5.1 ========= Properties ========== $ rhc cartridge add -c postgresql-8.4 -a mezz Similar for PostgreSQL 100 Sunday, March 17, 13
  51. GIT PUSH TO DEPLOY $ git push Counting objects: 5,

    done. Delta compression using up to 2 threads. Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 498 bytes, done. Total 3 (delta 1), reused 0 (delta 0) remote: restart_on_add=false remote: Waiting for stop to finish remote: Done remote: restart_on_add=false remote: ~/git/mezz.git ~/git/mezz.git remote: ~/git/mezz.git remote: Running .openshift/action_hooks/pre_build remote: setup.py found. Setting up virtualenv remote: New python executable in /var/lib/openshift/x/python-2.6/virtenv/bin/python remote: Installing setuptools............done. remote: Installing pip...............done. ... 101 Sunday, March 17, 13
  52. MORE RESOURCES • Getting started with Django on OpenShift https://openshift.redhat.com/community/get-started/django

    • openshift-django-example https://bitbucket.org/mdoglio/openshift-django-sample • Rapid Python and Django app deployment https://openshift.redhat.com/community/blogs/rapid-python-and-django-app- deployment-to-the-cloud-with-a-paas 102 Sunday, March 17, 13
  53. .elasticbeanstalk directory to store config files wsgi.py file config file

    with database settings for RDS 105 Sunday, March 17, 13
  54. ANATOMY OF GAE lib directory needed since GAE can’t pip

    install -r requirements.txt modified settings file: add lib dir to python path and DB settings config file which defines libraries and static files mappings 109 Sunday, March 17, 13
  55. Stackato OpenShift Dotcloud Heroku Python 2.7, 3.2 stackato runtimes 2.6

    (2.7) 2.6.5, 2.7.2, 3.1.2, 3.2.2 2.7.2 PostgreSQL 9.1 8.4 9.0 9.1.6 MySQL 5.5 5.1 5.1 (Yes, via RDS) Persisted FS Yes Yes Yes (Yes, via S3) Redis Yes, 2.4 No Yes, 2.4.11 (Yes, via addon) MongoDB Yes, 2.0 Yes, 2.2 Yes, 2.2.1 (Yes, via addon) Memcached Yes, 1.4 No Yes (Yes, via addon) RabbitMQ Yes, 2.4 No Yes, 2.8.5 (Yes, via addon) Solr No No Yes, 3.4.0 (Yes, via Websolr) Cron Yes Yes Yes Yes Extensible Yes, apt-get install Yes, DIY cartridge Yes, custom service Yes, buildpacks WebSockets Yes Yes Yes Yes, via Pusher add-on 114 Sunday, March 17, 13
  56. Google App Engine Elastic Beanstalk Python 2.7 2.6 PostgreSQL No

    No MySQL 5.5 via Cloud SQL Yes, via RDS Persisted FS Yes, via Blob storage Yes, via S3 Redis No No MongoDB No No Memcached Yes Yes, ElasticCache RabbitMQ No (SQS) Solr No (CloudSearch) Cron Yes, via Task Queues No Extensible No Yes, via AMI WebSockets Yes, via Channel API No 115 Sunday, March 17, 13
  57. WHY NOT PAAS? • Already invested in your own infrastructure.

    • Need to run on servers outside U.S. • Special requirements not met by PaaS services 116 Sunday, March 17, 13
  58. MORE INFO • Wrap-up from PaaS bake-off http://appsembler.com/blog/wrap-up-from-paas-bake-off/ • Django

    deployment using PaaS http://appsembler.com/blog/django-deployment-using-paas/ • django-deployer https://github.com/natea/django-deployer • paasbakeoff - code examples https://github.com/appsembler/paasbakeoff/ 122 Sunday, March 17, 13