Opening Open Source With DevOps

Transcript

1. Opening
   Open Source
   with
   DevOps
   @pimterry
   

   View Slide

2. View Slide

3. Source
   Open
   

   View Slide

4. Open SOURCE
   is powerful
   

   View Slide

5. Open SOURCE
   is hard
   

   View Slide

6. Open SOURCE is hard
   

   View Slide

7. Open SOURCE is hard
   

   View Slide

8. Open SOURCE is hard
   

   View Slide

9. Open SOURCE
   is hard
   

   View Slide

10. DevOps
    Enter
    

    View Slide

11. Nervous Beginners
    Focus on the challenges of
    

    View Slide

12. “Does my
    change
    break
    things?”
    

    View Slide

13. Sequelize
    

    View Slide

14. “Sequelize is a promise-based ORM
    for Node.js and io.js.
    It supports the dialects PostgreSQL,
    MySQL, MariaDB, SQLite and MSSQL
    and features solid transaction support,
    relations, read replication and more.”
    

    View Slide

15. Anything ?
    “But how can I change
    ”
    

    View Slide

16. Docker
    Enter
    

    View Slide

17. version: '2'
    services:
    # PostgreSQL
    postgres-95:
    image: camptocamp/postgis:9.5
    environment:
    POSTGRES_USER: sequelize_test
    POSTGRES_PASSWORD: sequelize_test
    POSTGRES_DB: sequelize_test
    ports:
    - "127.0.0.1:8998:5432"
    container_name: postgres-95
    # MySQL
    mysql-57:
    image: mysql:5.7
    environment:
    MYSQL_ROOT_PASSWORD: lollerskates
    MYSQL_DATABASE: sequelize_test
    MYSQL_USER: sequelize_test
    MYSQL_PASSWORD: sequelize_test
    ports:
    - "127.0.0.1:8999:3306"
    container_name: mysql-57
    docker-compose.yml
    

    View Slide

18. $ docker-compose up postgres-95 mysql-57
    Start environment
    

    View Slide

19. $ npm run test-docker
    Test everything
    

    View Slide

20. “How do I run it?”
    

    View Slide

21. 24 Pull Requests
    

    View Slide

22. TODO
    24 Pull Requests
    screenshot
    

    View Slide

23. 24 Pull Requests
    Install rbenv and ruby-build
    Install Ruby 2.3.1
    Install Postgres (and libpq)
    Configure Postgres
    Install Node.js and PhantomJS
    Install gem dependencies
    Create the required databases
    Run all the migrations for them
    Load the sample data
    Set up mailcatcher for email
    Start the server
    Local Dev Setup
    

    View Slide

24. 24 Pull Requests
    Bonus:
    now debug it
    

    View Slide

25. 24 Pull Requests
    Or just
    Panic
    

    View Slide

26. Ansible
    Enter
    

    View Slide

27. ---
    - hosts: all
    vars:
    timezone: UTC
    sys_packages:
    - python-software-properties
    - build-essential
    - libssl-dev
    - libreadline-dev
    - libpq-dev
    vars_files:
    - vars/pgsql.yml
    roles:
    - init
    - ruby
    - pgsql
    - app
    24 Pull Requests
    playbook.yml
    (simplified)
    

    View Slide

28. ---
    - hosts: all
    vars:
    timezone: UTC
    sys_packages:
    - python-software-properties
    - build-essential
    - libssl-dev
    - libreadline-dev
    - libpq-dev
    vars_files:
    - vars/pgsql.yml
    roles:
    - init
    - ruby
    - pgsql
    - app
    playbook.yml
    (simplified)
    24 Pull Requests
    

    View Slide

29. ---
    - name: Add ppa Repository - Node
    apt_repository: repo=ppa:chris-lea/node.js
    - name: Install NodeJS and PhantomJS
    apt: pkg={{ item }} state=latest
    with_items:
    - nodejs
    - phantomjs
    - name: Install Gems
    sudo: false
    shell: bundler install --path vendor/bundle
    - name: Rake db:create
    sudo: false
    shell: bundle exec rake db:create:all
    - name: Rake db:migrate
    sudo: false
    shell: bundle exec rake db:migrate
    - name: Feed DB
    sudo: false
    shell: bundle exec rake db:seed
    roles/app/tasks/main.yml
    (simplified)
    24 Pull Requests
    

    View Slide

30. $ ansible-playbook playbook.yml
    Local Dev Setup
    24 Pull Requests
    

    View Slide

31. Vagrant
    Enter
    

    View Slide

32. Vagrant.configure("2") do |config|
    config.vm.provider :virtualbox do |v|
    v.name = "TFPullRequests"
    v.customize ["modifyvm", :id, "--memory", 1024]
    end
    config.vm.box = "ubuntu/trusty64"
    config.vm.network :private_network, ip: "192.168.12.34"
    config.ssh.forward_agent = true
    if which('ansible-playbook')
    config.vm.provision "ansible" do |ansible|
    ansible.playbook = "ansible/playbook.yml"
    ansible.inventory_path = "ansible/inventories/dev"
    ansible.limit = 'all'
    end
    else
    config.vm.provision :shell, path: "ansible/windows.sh"
    end
    config.vm.synced_folder "./", "/vagrant", type: "nfs"
    end
    24 Pull Requests
    Vagrantfile
    (simplified)
    

    View Slide

33. $ vagrant up
    Local Dev Setup
    24 Pull Requests
    

    View Slide

34. “How do I
    ship this?”
    

    View Slide

35. View Slide

36. View Slide

37. Install rbenv and ruby-build
    Install Ruby 2.1
    Install MySQL
    Configure MySQL
    Create DB and app user in MySQL
    Clone codebase from Github
    Bundle install dependencies
    Fill out Staytus’s config files
    Run rake tasks to set up database
    schema
    Start the server
    Setting up Staytus
    

    View Slide

38. Install rbenv and ruby-build
    Install Ruby 2.1
    Install MySQL
    Configure MySQL
    Create DB and app user in MySQL
    Clone codebase from Github
    Bundle install dependencies
    Fill out Staytus’s config files
    Run rake tasks to set up database
    schema
    Start the server
    Setting up Staytus
    

    View Slide

39. Docker
    Enter
    (again)
    

    View Slide

40. Dockerfile
    (simplified)
    FROM ruby
    # Add MySQL to this image (generally not good practice!)
    RUN apt-get update && \
    export DEBIAN_FRONTEND=noninteractive && \
    echo mysql-server mysql-server/root_password \
    password temp-password | \
    debconf-set-selections && \
    echo mysql-server mysql-server/root_password_again \
    password temp-password | \
    debconf-set-selections && \
    apt-get install -y mysql-server nodejs
    COPY . /opt/staytus
    RUN cd /opt/staytus && \
    bundle install --deployment --without development:test
    ENTRYPOINT /opt/staytus/docker-start.sh
    VOLUME /var/lib/mysql
    VOLUME /opt/staytus/persisted
    EXPOSE 5000
    

    View Slide

41. #!/bin/bash
    /etc/init.d/mysql start
    cd /opt/staytus
    if [ ! -f /opt/staytus/persisted/config/database.yml ]; then
    # Configure DB with random password, if not already configured
    export RANDOM_PASSWORD=`openssl rand -base64 32`
    mysqladmin -u root -ptemp-password password $RANDOM_PASSWORD
    echo "CREATE DATABASE staytus COLLATE utf8_unicode_ci" | \
    mysql -u root -p$RANDOM_PASSWORD
    cp config/database.example.yml config/database.yml
    sed -i "s/username:.*/username: root/" config/database.yml
    sed -i "s|password:.*|password: $RANDOM_PASSWORD|" config/database.yml
    mkdir /opt/staytus/persisted/config
    cp config/database.yml /opt/staytus/persisted/config/database.yml
    bundle exec rake staytus:build staytus:install
    else
    cp /opt/staytus/persisted/config/database.yml config/database.yml
    # If already configured, check if there are any migrations to run
    bundle exec rake staytus:build staytus:upgrade
    fi
    bundle exec foreman start
    docker-start.sh
    (simplified)
    

    View Slide

42. $ docker run -d -p 0.0.0.0:80:5000
    --name=staytus adamcooke/staytus
    Set up a Staytus Site
    

    View Slide

43. $ curl -sS https://rawgit.com/galexrt/staytus/master/docker-compose.yml | \
    docker-compose -f - up -d
    Set up a Staytus Site
    (if moved to docker-compose)
    

    View Slide

44. “What if I don’t want
    to install Docker?”
    

    View Slide

45. View Slide

46. View Slide

47. CHEF
    Enter
    Omnibus
    

    View Slide

48. maintainer "GitLab Inc. "
    homepage "https://about.gitlab.com/"
    install_dir "/opt/gitlab"
    build_version Omnibus::BuildVersion.new.semver
    build_iteration Gitlab::BuildIteration.new.build_iteration
    override :ruby, version: '2.1.8', source: { md5: '091b62f0...' }
    override :rubygems, version: 'v2.5.1'
    override :pip, version: '7.1.2', source: { md5: '3823d234...' }
    override :redis, version: '2.8.24', source: { md5: '7b6eb6...' }
    override :postgresql, version: '9.2.1', source: { md5: '...' }
    override :liblzma, version: '5.2.2', source: { md5: '7cf6a...' }
    runtime_dependency "openssh-server"
    dependency "redis"
    dependency "nginx"
    dependency "remote-syslog" if ee
    dependency "nodejs"
    dependency "gitlab-shell"
    dependency "gitlab-cookbooks"
    dependency "gitlab-selinux"
    dependency "gitlab-config-template"
    dependency "mattermost"
    exclude "\.git*"
    package_user 'root'
    package_group 'root'
    config/projects/gitlab.rb
    (simplified)
    

    View Slide

49. name "nginx"
    default_version "1.9.10"
    dependency "pcre"
    dependency "openssl"
    source url: "http://nginx.org/download/nginx-#{version}.tar.gz",
    md5: "64cc970988356a5e0fc4fcd1ab84fe57"
    relative_path "nginx-#{version}"
    build do
    command ["./configure",
    "--prefix=#{install_dir}/embedded",
    "--with-http_ssl_module",
    "--with-http_gzip_static_module",
    "--with-http_v2_module",
    "--with-http_realip_module",
    "--with-ipv6",
    "--with-debug"].join(" ")
    command "make -j #{workers}",
    :env => {"LD_RUN_PATH" => "#{install_dir}/embedded/lib"}
    command "make install"
    end
    config/software/nginx.rb
    (simplified)
    

    View Slide

50. Install Gitlab
    

    View Slide

51. Ubuntu/Debian/Raspbian
    $ apt-get install gitlab-ce
    CentOS/RedHat
    $ yum install gitlab-ce
    OpenSUSE
    $ zypper install gitlab-ce
    Install Gitlab
    

    View Slide

52. View Slide

53. How can DevOps make Open Source more open?
    Create Confidence inspiring Test Environments
    Make developer setup effortless
    Distribute painless setup tools
    ...and much much more
    

    View Slide

54. Opening
    Open Source
    with
    DevOps
    @pimterry
    

    View Slide