Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Opening Open Source With DevOps

Opening Open Source With DevOps

Tim Perry

May 26, 2016
Tweet

More Decks by Tim Perry

Other Decks in Technology

Transcript

  1. Opening
    Open Source
    with
    DevOps
    @pimterry

    View Slide

  2. View Slide

  3. Source
    Open

    View Slide

  4. Open SOURCE
    is powerful

    View Slide

  5. Open SOURCE
    is hard

    View Slide

  6. Open SOURCE is hard

    View Slide

  7. Open SOURCE is hard

    View Slide

  8. Open SOURCE is hard

    View Slide

  9. Open SOURCE
    is hard

    View Slide

  10. DevOps
    Enter

    View Slide

  11. Nervous Beginners
    Focus on the challenges of

    View Slide

  12. “Does my
    change
    break
    things?”

    View Slide

  13. Sequelize

    View Slide

  14. “Sequelize is a promise-based ORM
    for Node.js and io.js.
    It supports the dialects PostgreSQL,
    MySQL, MariaDB, SQLite and MSSQL
    and features solid transaction support,
    relations, read replication and more.”

    View Slide

  15. Anything ?
    “But how can I change

    View Slide

  16. Docker
    Enter

    View Slide

  17. version: '2'
    services:
    # PostgreSQL
    postgres-95:
    image: camptocamp/postgis:9.5
    environment:
    POSTGRES_USER: sequelize_test
    POSTGRES_PASSWORD: sequelize_test
    POSTGRES_DB: sequelize_test
    ports:
    - "127.0.0.1:8998:5432"
    container_name: postgres-95
    # MySQL
    mysql-57:
    image: mysql:5.7
    environment:
    MYSQL_ROOT_PASSWORD: lollerskates
    MYSQL_DATABASE: sequelize_test
    MYSQL_USER: sequelize_test
    MYSQL_PASSWORD: sequelize_test
    ports:
    - "127.0.0.1:8999:3306"
    container_name: mysql-57
    docker-compose.yml

    View Slide

  18. $ docker-compose up postgres-95 mysql-57
    Start environment

    View Slide

  19. $ npm run test-docker
    Test everything

    View Slide

  20. “How do I run it?”

    View Slide

  21. 24 Pull Requests

    View Slide

  22. TODO
    24 Pull Requests
    screenshot

    View Slide

  23. 24 Pull Requests
    Install rbenv and ruby-build
    Install Ruby 2.3.1
    Install Postgres (and libpq)
    Configure Postgres
    Install Node.js and PhantomJS
    Install gem dependencies
    Create the required databases
    Run all the migrations for them
    Load the sample data
    Set up mailcatcher for email
    Start the server
    Local Dev Setup

    View Slide

  24. 24 Pull Requests
    Bonus:
    now debug it

    View Slide

  25. 24 Pull Requests
    Or just
    Panic

    View Slide

  26. Ansible
    Enter

    View Slide

  27. ---
    - hosts: all
    vars:
    timezone: UTC
    sys_packages:
    - python-software-properties
    - build-essential
    - libssl-dev
    - libreadline-dev
    - libpq-dev
    vars_files:
    - vars/pgsql.yml
    roles:
    - init
    - ruby
    - pgsql
    - app
    24 Pull Requests
    playbook.yml
    (simplified)

    View Slide

  28. ---
    - hosts: all
    vars:
    timezone: UTC
    sys_packages:
    - python-software-properties
    - build-essential
    - libssl-dev
    - libreadline-dev
    - libpq-dev
    vars_files:
    - vars/pgsql.yml
    roles:
    - init
    - ruby
    - pgsql
    - app
    playbook.yml
    (simplified)
    24 Pull Requests

    View Slide

  29. ---
    - name: Add ppa Repository - Node
    apt_repository: repo=ppa:chris-lea/node.js
    - name: Install NodeJS and PhantomJS
    apt: pkg={{ item }} state=latest
    with_items:
    - nodejs
    - phantomjs
    - name: Install Gems
    sudo: false
    shell: bundler install --path vendor/bundle
    - name: Rake db:create
    sudo: false
    shell: bundle exec rake db:create:all
    - name: Rake db:migrate
    sudo: false
    shell: bundle exec rake db:migrate
    - name: Feed DB
    sudo: false
    shell: bundle exec rake db:seed
    roles/app/tasks/main.yml
    (simplified)
    24 Pull Requests

    View Slide

  30. $ ansible-playbook playbook.yml
    Local Dev Setup
    24 Pull Requests

    View Slide

  31. Vagrant
    Enter

    View Slide

  32. Vagrant.configure("2") do |config|
    config.vm.provider :virtualbox do |v|
    v.name = "TFPullRequests"
    v.customize ["modifyvm", :id, "--memory", 1024]
    end
    config.vm.box = "ubuntu/trusty64"
    config.vm.network :private_network, ip: "192.168.12.34"
    config.ssh.forward_agent = true
    if which('ansible-playbook')
    config.vm.provision "ansible" do |ansible|
    ansible.playbook = "ansible/playbook.yml"
    ansible.inventory_path = "ansible/inventories/dev"
    ansible.limit = 'all'
    end
    else
    config.vm.provision :shell, path: "ansible/windows.sh"
    end
    config.vm.synced_folder "./", "/vagrant", type: "nfs"
    end
    24 Pull Requests
    Vagrantfile
    (simplified)

    View Slide

  33. $ vagrant up
    Local Dev Setup
    24 Pull Requests

    View Slide

  34. “How do I
    ship this?”

    View Slide

  35. View Slide

  36. View Slide

  37. Install rbenv and ruby-build
    Install Ruby 2.1
    Install MySQL
    Configure MySQL
    Create DB and app user in MySQL
    Clone codebase from Github
    Bundle install dependencies
    Fill out Staytus’s config files
    Run rake tasks to set up database
    schema
    Start the server
    Setting up Staytus

    View Slide

  38. Install rbenv and ruby-build
    Install Ruby 2.1
    Install MySQL
    Configure MySQL
    Create DB and app user in MySQL
    Clone codebase from Github
    Bundle install dependencies
    Fill out Staytus’s config files
    Run rake tasks to set up database
    schema
    Start the server
    Setting up Staytus

    View Slide

  39. Docker
    Enter
    (again)

    View Slide

  40. Dockerfile
    (simplified)
    FROM ruby
    # Add MySQL to this image (generally not good practice!)
    RUN apt-get update && \
    export DEBIAN_FRONTEND=noninteractive && \
    echo mysql-server mysql-server/root_password \
    password temp-password | \
    debconf-set-selections && \
    echo mysql-server mysql-server/root_password_again \
    password temp-password | \
    debconf-set-selections && \
    apt-get install -y mysql-server nodejs
    COPY . /opt/staytus
    RUN cd /opt/staytus && \
    bundle install --deployment --without development:test
    ENTRYPOINT /opt/staytus/docker-start.sh
    VOLUME /var/lib/mysql
    VOLUME /opt/staytus/persisted
    EXPOSE 5000

    View Slide

  41. #!/bin/bash
    /etc/init.d/mysql start
    cd /opt/staytus
    if [ ! -f /opt/staytus/persisted/config/database.yml ]; then
    # Configure DB with random password, if not already configured
    export RANDOM_PASSWORD=`openssl rand -base64 32`
    mysqladmin -u root -ptemp-password password $RANDOM_PASSWORD
    echo "CREATE DATABASE staytus COLLATE utf8_unicode_ci" | \
    mysql -u root -p$RANDOM_PASSWORD
    cp config/database.example.yml config/database.yml
    sed -i "s/username:.*/username: root/" config/database.yml
    sed -i "s|password:.*|password: $RANDOM_PASSWORD|" config/database.yml
    mkdir /opt/staytus/persisted/config
    cp config/database.yml /opt/staytus/persisted/config/database.yml
    bundle exec rake staytus:build staytus:install
    else
    cp /opt/staytus/persisted/config/database.yml config/database.yml
    # If already configured, check if there are any migrations to run
    bundle exec rake staytus:build staytus:upgrade
    fi
    bundle exec foreman start
    docker-start.sh
    (simplified)

    View Slide

  42. $ docker run -d -p 0.0.0.0:80:5000
    --name=staytus adamcooke/staytus
    Set up a Staytus Site

    View Slide

  43. $ curl -sS https://rawgit.com/galexrt/staytus/master/docker-compose.yml | \
    docker-compose -f - up -d
    Set up a Staytus Site
    (if moved to docker-compose)

    View Slide

  44. “What if I don’t want
    to install Docker?”

    View Slide

  45. View Slide

  46. View Slide

  47. CHEF
    Enter
    Omnibus

    View Slide

  48. maintainer "GitLab Inc. "
    homepage "https://about.gitlab.com/"
    install_dir "/opt/gitlab"
    build_version Omnibus::BuildVersion.new.semver
    build_iteration Gitlab::BuildIteration.new.build_iteration
    override :ruby, version: '2.1.8', source: { md5: '091b62f0...' }
    override :rubygems, version: 'v2.5.1'
    override :pip, version: '7.1.2', source: { md5: '3823d234...' }
    override :redis, version: '2.8.24', source: { md5: '7b6eb6...' }
    override :postgresql, version: '9.2.1', source: { md5: '...' }
    override :liblzma, version: '5.2.2', source: { md5: '7cf6a...' }
    runtime_dependency "openssh-server"
    dependency "redis"
    dependency "nginx"
    dependency "remote-syslog" if ee
    dependency "nodejs"
    dependency "gitlab-shell"
    dependency "gitlab-cookbooks"
    dependency "gitlab-selinux"
    dependency "gitlab-config-template"
    dependency "mattermost"
    exclude "\.git*"
    package_user 'root'
    package_group 'root'
    config/projects/gitlab.rb
    (simplified)

    View Slide

  49. name "nginx"
    default_version "1.9.10"
    dependency "pcre"
    dependency "openssl"
    source url: "http://nginx.org/download/nginx-#{version}.tar.gz",
    md5: "64cc970988356a5e0fc4fcd1ab84fe57"
    relative_path "nginx-#{version}"
    build do
    command ["./configure",
    "--prefix=#{install_dir}/embedded",
    "--with-http_ssl_module",
    "--with-http_gzip_static_module",
    "--with-http_v2_module",
    "--with-http_realip_module",
    "--with-ipv6",
    "--with-debug"].join(" ")
    command "make -j #{workers}",
    :env => {"LD_RUN_PATH" => "#{install_dir}/embedded/lib"}
    command "make install"
    end
    config/software/nginx.rb
    (simplified)

    View Slide

  50. Install Gitlab

    View Slide

  51. Ubuntu/Debian/Raspbian
    $ apt-get install gitlab-ce
    CentOS/RedHat
    $ yum install gitlab-ce
    OpenSUSE
    $ zypper install gitlab-ce
    Install Gitlab

    View Slide

  52. View Slide

  53. How can DevOps make Open Source more open?
    Create Confidence inspiring Test Environments
    Make developer setup effortless
    Distribute painless setup tools
    ...and much much more

    View Slide

  54. Opening
    Open Source
    with
    DevOps
    @pimterry

    View Slide