Provisioning virtualized infrastructure with Gradle Benjamin Muschko Principal Engineer, Gradleware @bmuschko

Manual infrastructure maintenance

Virtualization and "Infrastructure as Code" to the rescue

Continuous Delivery Demo Creation Environment Parity Dev Environment

Developer machine Infrastructure code edit commit pull Proper configuration management

Developer machine Infrastructure code VMs edit provision Standardized, consolidated environments

Easy to create and revert back to "good" known state Developer machine Initial provision Changed modify Destroyed tear down

Virtual Machines versus containers Hardware Host OS VM Guest OS Processes Binaries VM Guest OS Processes Binaries VM Guest OS Processes Binaries VM Guest OS Processes Binaries Hardware Host OS Binaries / Libraries Container Container Container

No content

Vagrant prerequisites Configures and bootstraps virtual machine Virtualization runtime environment +

send command manages Vagrant architecture Provisioner Vagrantfile invokes VM Developer machine Public Vagrant box catalog pull box share box

Vagrant quickstart $ vagrant up $ vagrant init hashicorp/precise64 $ vagrant ssh 1 2 3 Creates Vagrantfile with initialized with box image Downloads box image and starts virtual machine SSH into running Vagrant virtual machine

Gradle Vagrant plugin composition vagrant vagrant-base ‣ adds custom task types ‣ validates Vagrant installation ‣ adds standard tasks to project ‣ configures default conventions ‣ exposes DSL applies

buildscript { repositories { jcenter() } dependencies { classpath 'org.gradle.api.plugins:gradle-vagrant-plugin:0.6' } } apply plugin: 'vagrant' vagrant { boxDir = file('~/dev/my-vagrant-box') } Using convention plugin gradlew vagrantUp vagrant Points to Vagrant box definition Adds Vagrant plugin to build

import org.gradle.api.plugins.vagrant.tasks.* ext { fusionBoxDir = file('fusion-box') fusionProvider = 'vmware_fusion' } task startFusionVm(type: VagrantUp) { description = 'Starts VM machine running on VMware Fusion.' group = 'Fusion VM' boxDir = fusionBoxDir provider = fusionProvider } task stopFusionVm(type: VagrantDestroy) { description = 'Stops VM machine running on VMware Fusion.' group = 'Fusion VM' boxDir = fusionBoxDir provider = fusionProvider } User-defined Vagrant tasks vagrant-base Stands up VM with VMware Fusion provider Tears down VM

import org.gradle.api.plugins.vagrant.tasks.VagrantSsh task vagrantEcho(type: VagrantSsh) { description = 'Runs remote SSH command in Vagrant box.' sshCommand = "echo 'hello'" dependsOn vagrantUp finalizedBy vagrantDestroy } Executing SSH commands in VM gradlew vagrantEcho Definition of SSH command Vagrant SSH custom task Starts/stops VM Start VM Execute SSH Stop VM 1 2 3

vagrant { boxDir = file('~/dev/my-vagrant-box') environmentVariables { variable 'IP', '192.168.1.33' } } Controlling logic with env. variables Defines key/value pairs Vagrant.configure("2") do |config| config.vm.network "private_network", ip: ENV['IP'] end dynamic values via env. variables

Let there be smoke!

Test source code organization Production source Unit test source Smoke test source

ext { vmIp = '192.168.1.33' tomcatPort = 8080 } sourceSets { smokeTest { groovy.srcDir file('src/smokeTest/groovy') resources.srcDir file('src/smokeTest/resources') compileClasspath = sourceSets.main.output + configurations.testRuntime runtimeClasspath = output + compileClasspath } } task smokeTest(type: Test) { testClassesDir = sourceSets.smokeTest.output.classesDir classpath = sourceSets.smokeTest.runtimeClasspath systemProperty 'vmIp', vmIp systemProperty 'tomcatPort', tomcatPort } Smoke tests definition Organizing test classes as dedicated source set Inject runtime parameters gradlew smokeTest

dependencies { compile 'org.codehaus.groovy:groovy-all:2.2.2' testCompile 'org.apache.httpcomponents:httpclient:4.3.3' testCompile 'org.spockframework:spock-core:0.7-groovy-2.0' } smokeTest.mustRunAfter vagrantUp task startVmAndVerify { dependsOn vagrantUp, smokeTest finalizedBy vagrantDestroy } Smoke tests as part of build lifecycle Spock and HttpClient libraries used for tests Run smoke test after standing up VM gradlew startVmAndVerify Start VM Smoke tests Stop VM 1 2 3

package org.gradle.vm.smoke import org.apache.http.HttpResponse import spock.lang.Unroll class TomcatHttpSmokeTest extends AbstractHttpSmokeTest { @Unroll def "Can resolve #name with URL #url"() { when: HttpConnectionParams params = new HttpConnectionParams(retries: 10) HttpResponse httpResponse = get(url, params) then: assert httpResponse.statusLine.statusCode == HttpURLConnection.HTTP_OK, "Failed to reach '$url'. Reason: $httpResponse.statusLine" where: name | url 'dashboard' | tomcatBaseUrl 'hello world' | "$tomcatBaseUrl/examples/servlets/servlet/HelloWorldExample" } } Testing Tomcat availability with Spock Provides heavy lifting of making HTTP calls Data driven testing

package org.gradle.vm.smoke import org.apache.http.client.methods.* import org.apache.http.impl.client.* import spock.lang.Specification abstract class AbstractHttpSmokeTest extends Specification { protected HttpResponse get(String url, String contextPath = null, Map queryParams = null, HttpConnectionParams httpConnectionParams) { CloseableHttpClient httpClient try { httpClient = buildHttpClient(httpConnectionParams) URI uri = buildURI(url, contextPath, queryParams) HttpGet httpGet = new HttpGet(uri) return httpClient.execute(httpGet) } finally { httpClient?.close() } } } Abstract class for HTTP-based tests Use retry handler Apache HTTPClient highly stable and configurable

import org.gradle.api.plugins.vagrant.tasks.VagrantSsh version = '1.5' task deployWebAppToVM(type: VagrantSsh) { description = 'Deploys web application.' sshCommand = "scp http://binrepo:9090/internal/myapp-${project.version}.war /opt/tomcat/webapps" dependsOn vagrantUp finalizedBy vagrantDestroy } Pull deployment of a web app gradlew deployWebAppToVM SCP web artifact from binary repository Start VM Deploy app Stop VM 1 2 3

Keep acceptance tests green!

task functionalTest(type: Test) { testClassesDir = sourceSets.functionalTest.output.classesDir classpath = sourceSets.functionalTest.runtimeClasspath reports.html.destination = file("$reports.html.destination/functional") reports.junitXml.destination = file("$reports.junitXml.destination/functional") systemProperty 'geb.env', 'firefox' systemProperty 'geb.build.reportsDir', reporting.file("$name/geb") mustRunAfter deployWebAppToVM dependsOn smokeTest } Automate most important use cases as functional tests gradlew functionalTest Only run tests after environment is deemed "functional"

No content

pull image send command manages Docker architecture build image Developer machine Public Docker image repository push image Local Repository Container Dockerfile uses image images

Docker quickstart $ docker run busybox /bin/echo hello world $ docker pull busybox 1 2 Pulls minimal Linux image from Docker repository Runs image in container Executes echo command

buildscript { repositories { jcenter() } dependencies { classpath 'org.gradle.api.plugins:gradle-docker-plugin:0.1' } } apply plugin: 'docker' docker { serverUrl = 'http://remote.docker.com:4243' } Using the Gradle Docker plugin docker Points to Docker server URL exposing remote API Adds Docker plugin to build

import org.gradle.api.plugins.docker.tasks.image.DockerBuildImage task buildMyAppImage(type: DockerBuildImage) { inputDir = file('docker/myapp') tag = 'bmuschko/myapp' } Building a Docker image Builds image from Dockerfile FROM ubuntu MAINTAINER Benjamin Muschko RUN echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/sources.list RUN apt-get update ... Dockerfile instructions

import org.gradle.api.plugins.docker.tasks.container.* task createMyAppContainer(type: DockerCreateContainer) { dependsOn buildMyAppImage imageId = 'bmuschko/myapp' } task startMyAppContainer(type: DockerStartContainer) { dependsOn createMyAppContainer targetContainerId { createMyAppContainer.getContainerId() } } task stopMyAppContainer(type: DockerStopContainer) { targetContainerId { createMyAppContainer.getContainerId() } } Run container with image docker Creates container Use container ID for starting /stopping container

Virtualization in build pipelines

Exemplary build pipeline with Docker Commit stage Image creation Autom. accept. testing Manual testing Release pull image publish artifact build & push image download artifact run container run container run container pull image pull image

References Vagrant http://www.vagrantup.com Gradle Vagrant Plugin https://github.com/bmuschko/gradle-vagrant-plugin Docker https://www.docker.io Gradle Docker Plugin https://github.com/bmuschko/gradle-docker-plugin

> gradle askQuestions :askQuestions BUILD SUCCESSFUL Total time: 300 secs