Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Continuous Delivery for Python Developers - PyCon 8

Continuous Delivery for Python Developers - PyCon 8

Continuous Delivery sounds easy in theory, but it’s hard to do in practice. There are myriads of things you can and should do to get your code delivered faster, reliably. We look at what we can do as Python developers, or as a small or mid-sized team to make the industrialized software development production chain come true.

See the original presentation at http://slides.com/bittner/pycon8-continuous-delivery-for-python-developers


Peter Bittner

April 07, 2017


  1. Continuous Delivery for Python Developers PyCon 8, 2017 live slides

    @ tinyurl.com/pycon8-cd
  2. Peter Bittner Developer (of people, companies, code) Co-founder Painless Software

    @peterbittner, django@bittner.it behave-django codeship-yaml sentrylogs django-apptemplates djangocms-maps
  3. Hypes are hype! ... and hype means crazy.

  4. Our lives, our pain.

  5. Agenda 1. What is CD? 2. Why do it? 3.

    How do it? 4. Put it into practice 5. Demo, Q&A, workshop
  6. What Continuous Delivery? is

  7. Hands-on coding (infrastructure) Start filling in backlogs (PO) Start doing

    meetings (SM) Show how PRs/MRs work (pairing) Work with PM + management Internal communication, discussions (change mgmt) Continuous Delivery? “ . As a coach you do: 30% 20% 15% 5% 10% 15% 5%
  8. (C-level management) hype Replacement for Scrum+Kanban? 1st principle of Agile

    Manifesto All automatic, just push code! TDD, BDD, Selenium, Docker Deployment button for business What Is It?
  9. Definition “ ... a set of practices and principles ...

    building, testing, and releasing software, faster and more frequently “ ... put release schedule in the hands of the business - not IT Text Source: https://painless.software/continuous-delivery
  10. It's Agile (Ever Since) Source: http://agilemanifesto.org/principles.html “ Satisfy the customer!

  11. TDD, BDD, Selenium Tests are fundamental for build pipeline. No

    automated verification w/o tests No safeguarding against regression No safety net against deployments that would take the site down TDD good practice BDD helpful for acceptance Selenium / performance !! risk !!
  12. Containers (Docker) Containers make process easier. Feature parity across environments

    Develop, Staging, Production identical Simplify deployments, no builds on hosts Simplify rollbacks Optional, but makes it easier Container images built by pipeline Push complete system
  13. Why Continuous Delivery after all?

  14. On-boarding developers takes days "Works on my machine" Not releasing

    enough features Only a few can trigger deployment "Never deploy on a Friday!" Move to a different hoster (cloud?) Does It Solve My Problems?
  15. The Definition Said Build features faster + more frequently Test

    changes faster + more frequently Release changes faster + more frequently Everyone (even the client) can release ... and Docker said Environment parity (production = staging = dev machine) Definition: https://painless.software/continuous-delivery
  16. That's the Plan So, in future, we will: 1. Clone

    a repo + run a project locally 2. Make, test + push changes 3. Get feedback from build server 4. Have changes reviewed + released All in a matter of minutes No fear to break anything Happy devs + clients
  17. How Continuous Delivery do we make happen?

  18. Version control Feature branching - maybe Code review, PRs /

    MRs A test runner, task runner Build server, CI server, pipeline Artifact storage, image registry Hosting Monitoring What Do We Need?
  19. Development Process Someone needs to ship it. push code open

    PR/MR approve build review feature production system branch server system
  20. What to Look at Someone needs to maintain it. Infrastructure

    as code Agent-less deployment Stick to standard procedures + popular tools (best practices) Make it work for everyone * Self-explanatory procedures Don't break conventions Make developers love it!
  21. Build Servers + Registries Someone needs to build it. Pipeline

    will build, push and deploy Pick a solution or combo you love to work with (GitHub+Travis+DockerHub, GitLab, ...) Hosted may be cheaper than self-hosted (TCO) Build speed, pleasant UI + workflow may pay off more than a smaller price Registries: http://alternativeto.net/software/docker-hub/ CI Services: http://alternativeto.net/software/travis-ci/
  22. Docker Hosting Someone needs to serve it. Decent Docker hosting

    supports `docker machine` drivers Regular (dedicated) hosting with `docker` and `docker-compose` also works Avoid manual setup (transient servers) Shortlist: https://blog.codeship.com/the-shortlist-of-docker-hosting/
  23. Monitoring Someone needs to babysit it. Screen resource utilization after

    deployment Some issues are only discovered in production Capture stack traces, identify bottlenecks Get user feedback, communicate w/ users Collect historical data Transparency: https://painless.software/transparency Monitoring: http://alternativeto.net/software/sentry/
  24. Now Continuous Delivery let's do with Python!

  25. Pythonic Choices Tox w/ pytest, Flake8, Pylint, behave uWSGI, Nginx,

    Compose README.rst, no docs Unit and BDD tests prepared GitHub+Travis CI / GitLab / Bitbucket Run project: `docker-compose up` `manage.py runserver` works too Run tests: `tox` or `pytest`
  26. .travis.yml language: python python: 3.5 env: matrix: - TOXENV=flake8 -

    TOXENV=py27 - TOXENV=py35 install: pip install tox script: tox deploy: # magic stuff here
  27. tox.ini [tox] envlist = flake8,py27,py35 skipsdist = true [flake8] exclude

    = .cache,.git,.tox,build max-line-length = 80 [pytest] addopts = --junitxml=unittests.xml --strict
  28. docker-compose.yml version: "2" services: webserver: build: ./config/webserver application: build: context:

    . dockerfile: ./config/application/Dockerfile user: application database: image: postgres
  29. uwsgi.ini [uwsgi] cheaper = 2 chmod-socket = 660 chown-socket =

    application:application enable-threads = True master = True processes = 16 python-autoreload = 1 socket = /run/project/uwsgi.sock wsgi-file = /opt/project/application/wsgi.py
  30. nginx.conf server { location / { include uwsgi_params; uwsgi_pass unix:///run/project/uwsgi.sock;

    } location /media { root /opt/project } location /static { root /opt/project; } }
  31. README.rst Project Name ============ Awesome demo website for PyCon 8.

    Getting Started --------------- To start developing on this project simply bring up the Docker setup: .. code-block:: bash docker-compose up --build -d docker-compose logs -f
  32. Don't Create Those Files! 1. Create a project skeleton (cookiecutter)

    2. Use existing cookiecutters 3. Contribute to the one you love best! 4. Do the hard stuff in public (it will pay off) Run `cookiecutter gh:painless-software/ painless-continuous-delivery` Then start working immediately!
  33. Grandpa nailed it, once again. Just awesome. I'll tell our

  34. None
  35. One More Thing ...

  36. None
  37. Demo Time! https://github.com/painless- software/painless-continuous-delivery https://painless.cloud

  38. Thank you! for your precious time Painless Software Less pain,

    more fun.