Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Modern Web Deployment

Modern Web Deployment

Modern web deployment strategies talk given at #NairobiTechWeek

James Mwai

April 28, 2016
Tweet

More Decks by James Mwai

Other Decks in Technology

Transcript

  1. Agenda • What is web deployment? • Version Control •

    Testing • Continuous Integration • Orchestration • Containers • Container Management • Monitoring
  2. Web Deployment It is the action of rolling out new

    code to introduce change into an environment that hosts your web application.
  3. Version Control • Version control records changes in your code

    over time allowing you to recall specific versions later. • You should keep a version of every change you make in your code. • This allows you to revert your web app to a previous mistake you made earlier if you make a mistake. • See how people are making changes to the project over time. • Generally makes collaboration easier when working as a team. • No modern web deployment can happen without version control. • Most popular version control tool is git. Some people also use mercurial.
  4. Testing • Testing is a very important part of modern

    web dev. Code is considered good if it has good tests written as well. • Testing helps ensure you don’t break existing code when you introduce features. • If you test that new feature properly you won’t likely break existing functionality which makes deployment easier. • Tests help improve documentation of your code since you are testing existing functionality. • Tests speed up web deployment because you have confidence your new changes do not break existing functionality. • A test unit should focus on one proving one bit of functionality. • Each test unit should be fully independent. Each test unit should be able to run independently regardless if when it’s called.
  5. Unit Vs Integration Tests • A unit test determines the

    correctness of an individual functionality in a large codebase. • A unit test should work as expected in isolation from other tests • Almost all languages have unit testing framework. • Integration testing tests the system as whole.
  6. Continuous Integration • Team members integrate code into a shared

    repository continuously. • CI automates the building, testing and deployment of the code. • Automatically run unit tests as code is checked in to detect problems early • CI dramatically reduces deployment time • Helps prevent deploying broken code into production.
  7. Open Source CI Tools • Jenkins - Most popular CI

    tool written in Java. Has plenty of plugins • Go - CD tool built by thoughtworks • Strider - Open Source CI tool written in Nodejs • BuildBot - Open Source Python CI • TeamCity - Another free self-hosted CD platform by JetBrains
  8. CI as a Service No need to setup or manage,

    just integrate with your code hosting service. • CodeShip • Travis CI • Circle CI • Gitlab CI • BuildKite
  9. CI Best Practises • Never go home on a broken

    build • Always run tests locally before committing. • Wait for the build process to finish before moving on. • Always be ready to revert to previous version of your code if a build fails. • Take responsibility if you break something.
  10. Orchestration • Automated arrangement, coordination, and management of computer systems,

    middleware and services. • Allows us to automate setup of web servers. • Automate modification of servers from an existing state to a desired state • Automate deployment of our web applications • Our infrastructure lives as part of our code • Will save your organisation time and money
  11. Some Tools of the trade • Ansible • Capistrano •

    Fabric • SaltStack • Chef • Puppet
  12. SaltStack • Scalable and flexible configuration management software for event-driven

    infrastructure automation. • Written in Python. You can extend it using Python • You have a Salt-Master which hosts all the configurations policies • Salt-Minions are the nodes in your infrastructure that you want managed. • Salt uses ZeroMQ to push configs between Master and minions • https://docs.saltstack.com
  13. Capistrano • Written in Ruby • Popular in the Rails

    community • Can be used with other languages as well • Organises operative directions in files known as recipes • Not just userful with deployment, you can use it to build servers, install applications, maintenance and monitoring • Great documentation at http://capistranorb.com/#
  14. Chef • Powerful configuration management system that can be used

    to programmatically control your infrastructure environment in a predictable manner • Also uses ruby and stores config details in files called recipes. • Chef Server: Central location for your configuration recipes, cookbooks, and node and workstation definitions. Other machines in your infrastructure use the chef server for deployment configuration. • Chef Nodes: Chef nodes are the deployment targets that are configured by Chef.
  15. Puppet • Open Source automation software from Puppet Labs •

    There is an enterprise versions as well • Describes machine configuration in a declarative language. • Config code is stored in files called resource declarations • Declarations are organised into manifests. • A collection of manifests and data are known as modules.
  16. Fabric • Python Library for command-line streamlining use of SSH.

    • Provide a host of operations for executing local and remote SSH commands. • You write your tasks as python functions. Store them in a fabfile.py file and fabric will execute them on demand. • It’s just a python library on top of ssh that allows you to store repetitive ssh tasks in python files. • Different from others in that it uses the push methodology while chef,ansible loads your configs and executes on the
  17. Ansible • According to the creators, Ansible is the simplest

    way to automate apps and IT infrastructure • Uses YAML syntax. • Configuration details are organised in roles with each roles containing it’s own files, tasks, handlers, vars, templates. • Playbooks contain roles and target hosts. • Plenty of free tested roles available on https://galaxy. ansible.com • Recently acquired by Redhat
  18. - name: Install Nginx sudo: yes apt: pkg=nginx state=installed update_cache=yes

    - name: Disable the default site sudo: yes file: path=/etc/nginx/sites-enabled/default state=absent - name: Install Nginx Configuration File. sudo: yes copy: src={{ project_name }} dest=/etc/nginx/sites-enabled/{{ project_name }} notify: Reload Nginx
  19. Containers • OS-Level-Virtualization of Linux allowing multiple user- space instances.

    • A container contains isolated process running independently in the LInux OS. • You can have many isolated containers running on the same Linux kernel each behaving like a standalone Linux OS. • You can prioritize CPU, IO, Memory, Network to individual containers.
  20. Docker • Most popular Container Management Service • Allows you

    to easily run your apps inside Linux Containers • Dockerized apps are completely portable and can run anywhere; Linux, OSX, Windows. • Packaging your web app in Docker means it will always work the same in any environment.
  21. Docker Concepts • Docker Image - Packaged application instance with

    all it’s software dependencies and the processes to run. • Docker Container - Instance of an image. We use a Docker image to create a Docker container. • Dockerfile - A file containing declarations on automating the image building process • Docker Compose - Organises your containers into logical units enabling them to communicate with each other. • Docker Hub - Docker container hosting
  22. #Docker Pombola Image Example FROM i686/ubuntu:14.04 MAINTAINER James Mwai <[email protected]>

    RUN apt-get update RUN apt-get install -y python-software-properties # + all other dependencies RUN pip install --upgrade pip RUN pip install --upgrade virtualenv RUN mkdir pombola_root && cd pombola_root RUN git clone https://github.com/mysociety/pombola.git
  23. Docker Swarm Native container clustering tool created by Docker. It

    uses the standard Docker API Turns a groups of Docker containers into a single virtual Docker engine You have a Swarm Manager which acts as the Master with each host running the Swarm Agent. Can scale up to 1000 nodes running 50000
  24. Kubernetes Open source orchestration system for Docker containers. Created by

    Google leveraging their internal experience using containers in production for the last decade. Containers are organised into logical units called pods. Kubernetes schedules your pods and grows them in your cluster
  25. Amazon ECS Elastic Compute Container Service. Highly scalable, fast, container

    management service for running Docker containers on Amazon EC2 Lets you launch, stop, schedule, docker containers with simple API calls. Containers are grouped into clusters deployed on a fleet of EC2 instances.
  26. Google Container Engine • Automated Container management platform • Easily

    run your Docker containers on Google Cloud • Powered by Kubernetes • Priced based on Compute Engine prices • Container engines schedules your containers onto clusters based on predefined rules that you set yourself. • You define containers requirements e.g CPU, Memory in a simple json config file
  27. Fleet Low-level cluster management tool from CoreOs. Allows you to

    deploy docker containers on arbitrary hosts on a cluster. Built on top of Systemd Whereas Systemd is a system and service for a single linux host, fleet extends this to a cluster of machines. Allows you to treat your cluster of machines as a single host.
  28. Apache Mesos Open source clustering engine started at UC Berkeley.

    Designed to scale to very large clusters running on thousands of hosts with high availability and resilience. Used by large companies with like Twitter, eBay, Airbnb.
  29. Monitoring your Web Apps Consistently monitor the health your web

    infrastructure. Monitoring helps ensure there is uptime, load times, fast response time... Monitoring ensures your app delivers functionality as expected. Good monitoring tools will alert you when there is a problem somewhere in your app or infrastructure. Monitoring will help you improve your apps performance.
  30. Logging and Metrics • Sentry - Real Time exception tracking

    for webapps. Started in the Django community. Now a paid service. • LogStash - Collect and analyze logs from various sources. • Kibana - Collect, analyze, visualize, search your logs. • Datadog - Collect and visualize various performance metrics from your apps.
  31. Are you a modern web dev? Can you make a

    build in one step? Do you make daily builds? Do you use configuration management tools to automate infrastructure? Is the development setup documented and automated? Can you rollout and rollback deployments in one step? Can applications be deployed to a new server setup without changes to the code? Can you tell the health status of your server(s) at glance.
  32. StackDriver • Monitor your cloud platform resources • Powerful monitoring,

    logging, metrics • Monitor GCP and AWS platforms. • Provides a wide variety of metrics; alerts, log monitoring, dashboards, health reporting, tracing capabilities. • Easily find and fix issues in your cloud infrastructure • Integrates easily with all Cloud services