Opening Open-Source with DevOps

by Tim Perry
DevOps Pro Vilnius 2016

Transcript

1. Opening Open Source with DevOps @pimterry
3. Source Open
4. Open SOURCE is powerful
5. Open SOURCE is hard
6. Open SOURCE is hard
7. Open SOURCE is hard
8. Open SOURCE is hard
9. Open SOURCE is hard
10. DevOps Enter
11. How can DevOps make more accessible? Open Source
12. Nervous Beginners Focus on the challenges of
13. “Does my change break things?”
14. Sequelize
15. “Sequelize is a promise-based ORM for Node.js and io.js. It supports the dialects PostgreSQL, MySQL, MariaDB, SQLite and MSSQL and features solid transaction support, relations, read replication and more.”
16. Anything ? “But how can I change ”
17. Docker Enter
18. FROM node:6 RUN apt-get install libpq-dev COPY package.json / ENV NPM_CONFIG_LOGLEVEL error RUN npm install WORKDIR /sequelize VOLUME /sequelize Dockerfile
19. sequelize: build: . links: - mysql - postgres volumes: - .:/sequelize environment: SEQ_DB: sequelize_test SEQ_USER: sequelize_test SEQ_PW: sequelize_test JSHINT: /node_modules/.bin/jshint MOCHA: /node_modules/.bin/mocha postgres: image: postgres:9.4 environment: POSTGRES_USER: sequelize_test POSTGRES_PASSWORD: sequelize_test mysql: image: mysql:5.6 environment: MYSQL_DATABASE: sequelize_test MYSQL_USER: sequelize_test MYSQL_PASSWORD: sequelize_test docker-compose.yml
20. $ docker-compose run sequelize \ /bin/sh -c “npm run test-all” Test everything
21. $ npm run test-docker Test everything
22. “How do I run it?”
23. 24 Pull Requests
24. TODO 24 Pull Requests screenshot
25. 24 Pull Requests Install rbenv and ruby-build Install Ruby 2.3.1 Install Postgres Configure Postgres Install Node.js and PhantomJS Install gem dependencies Create the required databases Run all the migrations for them Load the sample data Set up mailcatcher for email Start the server Local Dev Setup
26. 24 Pull Requests Bonus: now debug it
27. 24 Pull Requests Or just Panic
28. Ansible Enter
29. --- - hosts: all vars: timezone: UTC sys_packages: - python-software-properties - build-essential - libssl-dev - libreadline-dev - libpq-dev vars_files: - vars/pgsql.yml roles: - init - ruby - pgsql - app 24 Pull Requests playbook.yml (simplified)
30. --- - hosts: all vars: timezone: UTC sys_packages: - python-software-properties - build-essential - libssl-dev - libreadline-dev - libpq-dev vars_files: - vars/pgsql.yml roles: - init - ruby - pgsql - app playbook.yml (simplified) 24 Pull Requests
31. --- - name: Add ppa Repository - Node apt_repository: repo=ppa:chris-lea/node.js - name: Install NodeJS and PhantomJS apt: pkg={{ item }} state=latest with_items: - nodejs - phantomjs - name: Install Gems sudo: false shell: bundler install --path vendor/bundle - name: Rake db:create sudo: false shell: bundle exec rake db:create:all - name: Rake db:migrate sudo: false shell: bundle exec rake db:migrate - name: Feed DB sudo: false shell: bundle exec rake db:seed roles/app/tasks/main.yml (simplified) 24 Pull Requests
32. $ ansible-playbook playbook.yml Local Dev Setup 24 Pull Requests
33. Vagrant Enter
34. Vagrant.configure("2") do |config| config.vm.provider :virtualbox do |v| v.name = "TFPullRequests" v.customize ["modifyvm", :id, "--memory", 1024] end config.vm.box = "ubuntu/trusty64" config.vm.network :private_network, ip: "192.168.12.34" config.ssh.forward_agent = true if which('ansible-playbook') config.vm.provision "ansible" do |ansible| ansible.playbook = "ansible/playbook.yml" ansible.inventory_path = "ansible/inventories/dev" ansible.limit = 'all' end else config.vm.provision :shell, path: "ansible/windows.sh" end config.vm.synced_folder "./", "/vagrant", type: "nfs" end 24 Pull Requests Vagrantfile (simplified)
35. $ vagrant up Local Dev Setup 24 Pull Requests
36. “How do I ship this?”
39. Install rbenv and ruby-build Install Ruby 2.1 Install MySQL Configure MySQL Create DB and app user in MySQL Clone codebase from Github Bundle install dependencies Fill out Staytus’s config files Run rake tasks to set up database schema Start the server Setting up Staytus
40. Install rbenv and ruby-build Install Ruby 2.1 Install MySQL Configure MySQL Create DB and app user in MySQL Clone codebase from Github Bundle install dependencies Fill out Staytus’s config files Run rake tasks to set up database schema Start the server Setting up Staytus
41. Docker Enter (again)
42. Dockerfile (simplified) FROM ruby USER root # Add MySQL to this image (generally not good practice!) RUN apt-get update && \ export DEBIAN_FRONTEND=noninteractive && \ echo mysql-server mysql-server/root_password \ password temp-password | \ debconf-set-selections && \ echo mysql-server mysql-server/root_password_again \ password temp-password | \ debconf-set-selections && \ apt-get install -y mysql-server nodejs COPY . /opt/staytus RUN cd /opt/staytus && \ bundle install --deployment --without development:test ENTRYPOINT /opt/staytus/docker-start.sh VOLUME /var/lib/mysql VOLUME /opt/staytus/persisted EXPOSE 5000
43. #!/bin/bash /etc/init.d/mysql start cd /opt/staytus if [ ! -f /opt/staytus/persisted/config/database.yml ]; then # Configure DB with random password, if not already configured export RANDOM_PASSWORD=`openssl rand -base64 32` mysqladmin -u root -ptemp-password password $RANDOM_PASSWORD echo "CREATE DATABASE staytus COLLATE utf8_unicode_ci" | \ mysql -u root -p$RANDOM_PASSWORD cp config/database.example.yml config/database.yml sed -i "s/username:.*/username: root/" config/database.yml sed -i "s|password:.*|password: $RANDOM_PASSWORD|" config/database.yml mkdir /opt/staytus/persisted/config cp config/database.yml /opt/staytus/persisted/config/database.yml bundle exec rake staytus:build staytus:install else cp /opt/staytus/persisted/config/database.yml config/database.yml # If already configured, check if there are any migrations to run bundle exec rake staytus:build staytus:upgrade fi bundle exec foreman start docker-start.sh (simplified)
44. $ docker run -d -p 0.0.0.0:80:5000 --name=staytus adamcooke/staytus Set up a Staytus Site
45. $ curl -sS https://rawgit.com/galexrt/staytus/master/docker-compose.yml | \ docker-compose -f - up -d Set up a Staytus Site (if moved to docker-compose)
46. “What if I don’t want to install Docker?”
49. CHEF Enter Omnibus
50. maintainer "GitLab Inc. <support@gitlab.com>" homepage "https://about.gitlab.com/" install_dir "/opt/gitlab" build_version Omnibus::BuildVersion.new.semver build_iteration Gitlab::BuildIteration.new.build_iteration override :ruby, version: '2.1.8', source: { md5: '091b62f0...' } override :rubygems, version: 'v2.5.1' override :pip, version: '7.1.2', source: { md5: '3823d234...' } override :redis, version: '2.8.24', source: { md5: '7b6eb6...' } override :postgresql, version: '9.2.1', source: { md5: '...' } override :liblzma, version: '5.2.2', source: { md5: '7cf6a...' } runtime_dependency "openssh-server" dependency "redis" dependency "nginx" dependency "remote-syslog" if ee dependency "nodejs" dependency "gitlab-shell" dependency "gitlab-cookbooks" dependency "gitlab-selinux" dependency "gitlab-config-template" dependency "mattermost" exclude "\.git*" package_user 'root' package_group 'root' config/projects/gitlab.rb (simplified)
51. name "nginx" default_version "1.9.12" dependency "pcre" dependency "openssl" source url: "http://nginx.org/download/nginx-#{version}.tar.gz", md5: "0afe4a7e589a0de43b7b54aa055a4351" relative_path "nginx-#{version}" build do command ["./configure", "--prefix=#{install_dir}/embedded", "--with-http_ssl_module", "--with-http_gzip_static_module", "--with-http_v2_module", "--with-http_realip_module", "--with-ipv6", "--with-debug"].join(" ") command "make install" end config/software/nginx.rb (simplified)
52. Ubuntu/Debian/Raspbian $ apt-get install gitlab-ce CentOS $ yum install gitlab-ce Install Gitlab
54. open-source developers Empower
55. How can DevOps make Open Source more accessible? Create Confidence inspiring Test Environments Make developer setup effortless Distribute painless setup tools
56. Opening Open Source with DevOps @pimterry