Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Opening Open Source With DevOps

Opening Open Source With DevOps

Tim Perry

May 26, 2016
Tweet

More Decks by Tim Perry

Other Decks in Technology

Transcript

  1. Opening
    Open Source
    with
    DevOps
    @pimterry

    View full-size slide

  2. Open SOURCE
    is powerful

    View full-size slide

  3. Open SOURCE
    is hard

    View full-size slide

  4. Open SOURCE is hard

    View full-size slide

  5. Open SOURCE is hard

    View full-size slide

  6. Open SOURCE is hard

    View full-size slide

  7. Open SOURCE
    is hard

    View full-size slide

  8. Nervous Beginners
    Focus on the challenges of

    View full-size slide

  9. “Does my
    change
    break
    things?”

    View full-size slide

  10. “Sequelize is a promise-based ORM
    for Node.js and io.js.
    It supports the dialects PostgreSQL,
    MySQL, MariaDB, SQLite and MSSQL
    and features solid transaction support,
    relations, read replication and more.”

    View full-size slide

  11. Anything ?
    “But how can I change

    View full-size slide

  12. version: '2'
    services:
    # PostgreSQL
    postgres-95:
    image: camptocamp/postgis:9.5
    environment:
    POSTGRES_USER: sequelize_test
    POSTGRES_PASSWORD: sequelize_test
    POSTGRES_DB: sequelize_test
    ports:
    - "127.0.0.1:8998:5432"
    container_name: postgres-95
    # MySQL
    mysql-57:
    image: mysql:5.7
    environment:
    MYSQL_ROOT_PASSWORD: lollerskates
    MYSQL_DATABASE: sequelize_test
    MYSQL_USER: sequelize_test
    MYSQL_PASSWORD: sequelize_test
    ports:
    - "127.0.0.1:8999:3306"
    container_name: mysql-57
    docker-compose.yml

    View full-size slide

  13. $ docker-compose up postgres-95 mysql-57
    Start environment

    View full-size slide

  14. $ npm run test-docker
    Test everything

    View full-size slide

  15. “How do I run it?”

    View full-size slide

  16. 24 Pull Requests

    View full-size slide

  17. TODO
    24 Pull Requests
    screenshot

    View full-size slide

  18. 24 Pull Requests
    Install rbenv and ruby-build
    Install Ruby 2.3.1
    Install Postgres (and libpq)
    Configure Postgres
    Install Node.js and PhantomJS
    Install gem dependencies
    Create the required databases
    Run all the migrations for them
    Load the sample data
    Set up mailcatcher for email
    Start the server
    Local Dev Setup

    View full-size slide

  19. 24 Pull Requests
    Bonus:
    now debug it

    View full-size slide

  20. 24 Pull Requests
    Or just
    Panic

    View full-size slide

  21. Ansible
    Enter

    View full-size slide

  22. ---
    - hosts: all
    vars:
    timezone: UTC
    sys_packages:
    - python-software-properties
    - build-essential
    - libssl-dev
    - libreadline-dev
    - libpq-dev
    vars_files:
    - vars/pgsql.yml
    roles:
    - init
    - ruby
    - pgsql
    - app
    24 Pull Requests
    playbook.yml
    (simplified)

    View full-size slide

  23. ---
    - hosts: all
    vars:
    timezone: UTC
    sys_packages:
    - python-software-properties
    - build-essential
    - libssl-dev
    - libreadline-dev
    - libpq-dev
    vars_files:
    - vars/pgsql.yml
    roles:
    - init
    - ruby
    - pgsql
    - app
    playbook.yml
    (simplified)
    24 Pull Requests

    View full-size slide

  24. ---
    - name: Add ppa Repository - Node
    apt_repository: repo=ppa:chris-lea/node.js
    - name: Install NodeJS and PhantomJS
    apt: pkg={{ item }} state=latest
    with_items:
    - nodejs
    - phantomjs
    - name: Install Gems
    sudo: false
    shell: bundler install --path vendor/bundle
    - name: Rake db:create
    sudo: false
    shell: bundle exec rake db:create:all
    - name: Rake db:migrate
    sudo: false
    shell: bundle exec rake db:migrate
    - name: Feed DB
    sudo: false
    shell: bundle exec rake db:seed
    roles/app/tasks/main.yml
    (simplified)
    24 Pull Requests

    View full-size slide

  25. $ ansible-playbook playbook.yml
    Local Dev Setup
    24 Pull Requests

    View full-size slide

  26. Vagrant
    Enter

    View full-size slide

  27. Vagrant.configure("2") do |config|
    config.vm.provider :virtualbox do |v|
    v.name = "TFPullRequests"
    v.customize ["modifyvm", :id, "--memory", 1024]
    end
    config.vm.box = "ubuntu/trusty64"
    config.vm.network :private_network, ip: "192.168.12.34"
    config.ssh.forward_agent = true
    if which('ansible-playbook')
    config.vm.provision "ansible" do |ansible|
    ansible.playbook = "ansible/playbook.yml"
    ansible.inventory_path = "ansible/inventories/dev"
    ansible.limit = 'all'
    end
    else
    config.vm.provision :shell, path: "ansible/windows.sh"
    end
    config.vm.synced_folder "./", "/vagrant", type: "nfs"
    end
    24 Pull Requests
    Vagrantfile
    (simplified)

    View full-size slide

  28. $ vagrant up
    Local Dev Setup
    24 Pull Requests

    View full-size slide

  29. “How do I
    ship this?”

    View full-size slide

  30. Install rbenv and ruby-build
    Install Ruby 2.1
    Install MySQL
    Configure MySQL
    Create DB and app user in MySQL
    Clone codebase from Github
    Bundle install dependencies
    Fill out Staytus’s config files
    Run rake tasks to set up database
    schema
    Start the server
    Setting up Staytus

    View full-size slide

  31. Install rbenv and ruby-build
    Install Ruby 2.1
    Install MySQL
    Configure MySQL
    Create DB and app user in MySQL
    Clone codebase from Github
    Bundle install dependencies
    Fill out Staytus’s config files
    Run rake tasks to set up database
    schema
    Start the server
    Setting up Staytus

    View full-size slide

  32. Docker
    Enter
    (again)

    View full-size slide

  33. Dockerfile
    (simplified)
    FROM ruby
    # Add MySQL to this image (generally not good practice!)
    RUN apt-get update && \
    export DEBIAN_FRONTEND=noninteractive && \
    echo mysql-server mysql-server/root_password \
    password temp-password | \
    debconf-set-selections && \
    echo mysql-server mysql-server/root_password_again \
    password temp-password | \
    debconf-set-selections && \
    apt-get install -y mysql-server nodejs
    COPY . /opt/staytus
    RUN cd /opt/staytus && \
    bundle install --deployment --without development:test
    ENTRYPOINT /opt/staytus/docker-start.sh
    VOLUME /var/lib/mysql
    VOLUME /opt/staytus/persisted
    EXPOSE 5000

    View full-size slide

  34. #!/bin/bash
    /etc/init.d/mysql start
    cd /opt/staytus
    if [ ! -f /opt/staytus/persisted/config/database.yml ]; then
    # Configure DB with random password, if not already configured
    export RANDOM_PASSWORD=`openssl rand -base64 32`
    mysqladmin -u root -ptemp-password password $RANDOM_PASSWORD
    echo "CREATE DATABASE staytus COLLATE utf8_unicode_ci" | \
    mysql -u root -p$RANDOM_PASSWORD
    cp config/database.example.yml config/database.yml
    sed -i "s/username:.*/username: root/" config/database.yml
    sed -i "s|password:.*|password: $RANDOM_PASSWORD|" config/database.yml
    mkdir /opt/staytus/persisted/config
    cp config/database.yml /opt/staytus/persisted/config/database.yml
    bundle exec rake staytus:build staytus:install
    else
    cp /opt/staytus/persisted/config/database.yml config/database.yml
    # If already configured, check if there are any migrations to run
    bundle exec rake staytus:build staytus:upgrade
    fi
    bundle exec foreman start
    docker-start.sh
    (simplified)

    View full-size slide

  35. $ docker run -d -p 0.0.0.0:80:5000
    --name=staytus adamcooke/staytus
    Set up a Staytus Site

    View full-size slide

  36. $ curl -sS https://rawgit.com/galexrt/staytus/master/docker-compose.yml | \
    docker-compose -f - up -d
    Set up a Staytus Site
    (if moved to docker-compose)

    View full-size slide

  37. “What if I don’t want
    to install Docker?”

    View full-size slide

  38. CHEF
    Enter
    Omnibus

    View full-size slide

  39. maintainer "GitLab Inc. "
    homepage "https://about.gitlab.com/"
    install_dir "/opt/gitlab"
    build_version Omnibus::BuildVersion.new.semver
    build_iteration Gitlab::BuildIteration.new.build_iteration
    override :ruby, version: '2.1.8', source: { md5: '091b62f0...' }
    override :rubygems, version: 'v2.5.1'
    override :pip, version: '7.1.2', source: { md5: '3823d234...' }
    override :redis, version: '2.8.24', source: { md5: '7b6eb6...' }
    override :postgresql, version: '9.2.1', source: { md5: '...' }
    override :liblzma, version: '5.2.2', source: { md5: '7cf6a...' }
    runtime_dependency "openssh-server"
    dependency "redis"
    dependency "nginx"
    dependency "remote-syslog" if ee
    dependency "nodejs"
    dependency "gitlab-shell"
    dependency "gitlab-cookbooks"
    dependency "gitlab-selinux"
    dependency "gitlab-config-template"
    dependency "mattermost"
    exclude "\.git*"
    package_user 'root'
    package_group 'root'
    config/projects/gitlab.rb
    (simplified)

    View full-size slide

  40. name "nginx"
    default_version "1.9.10"
    dependency "pcre"
    dependency "openssl"
    source url: "http://nginx.org/download/nginx-#{version}.tar.gz",
    md5: "64cc970988356a5e0fc4fcd1ab84fe57"
    relative_path "nginx-#{version}"
    build do
    command ["./configure",
    "--prefix=#{install_dir}/embedded",
    "--with-http_ssl_module",
    "--with-http_gzip_static_module",
    "--with-http_v2_module",
    "--with-http_realip_module",
    "--with-ipv6",
    "--with-debug"].join(" ")
    command "make -j #{workers}",
    :env => {"LD_RUN_PATH" => "#{install_dir}/embedded/lib"}
    command "make install"
    end
    config/software/nginx.rb
    (simplified)

    View full-size slide

  41. Install Gitlab

    View full-size slide

  42. Ubuntu/Debian/Raspbian
    $ apt-get install gitlab-ce
    CentOS/RedHat
    $ yum install gitlab-ce
    OpenSUSE
    $ zypper install gitlab-ce
    Install Gitlab

    View full-size slide

  43. How can DevOps make Open Source more open?
    Create Confidence inspiring Test Environments
    Make developer setup effortless
    Distribute painless setup tools
    ...and much much more

    View full-size slide

  44. Opening
    Open Source
    with
    DevOps
    @pimterry

    View full-size slide