Powering a build pipeline with Docker and Jenkins

Powering a build pipeline with Docker and Jenkins

Continuous Delivery is key to delivering software fast and frequently. Jenkins 2 has made great strides in providing a solution for modeling a build pipeline as code. In addition, Docker invoked from a Gradle build can help with implementing certain aspects of your pipeline. Combining the benefits of these tools makes it possible to implement a streamlined and bulletproof process.

In this demo-driven session, you will learn how to construct and operate a declarative pipeline with Jenkins. You’ll see a Spring Boot application taken all the way through the pipeline, from building the Docker image to pushing the image to a registry, using it for integration testing, and finally deploying the application to a Docker Swarm or Kubernetes.

8f2248c6bfcc6df39a2cd8edf4267cb5?s=128

Benjamin Muschko

September 16, 2019
Tweet

Transcript

  1. 5.

    Agenda Typical Continuous Delivery uses cases How can Docker help?

    Show me some code! The glue as code in Jenkins
  2. 17.
  3. 24.

    Dockerfile for Spring Boot app FROM openjdk:jre-alpine COPY todo-webservice-1.0.jar↵ /app/todo-webservice-1.0.jar


    ENTRYPOINT ["java"]
 CMD ["-jar", "/app/todo-webservice-1.0.jar"]
 HEALTHCHECK CMD wget --quiet --tries=1 --spider↵ http://localhost:8080/actuator/health || exit 1
 EXPOSE 8080
  4. 25.

    Dockerfile with layers for app files FROM openjdk:jre-alpine WORKDIR /app

    COPY libs libs/ COPY resources resources/ COPY classes classes/ ENTRYPOINT ["java", "-cp",↵ "/app/resources:/app/classes:/app/libs/*",↵ "com.bmuschko.MyApplication"]
 HEALTHCHECK CMD wget --quiet --tries=1 --spider↵ http://localhost:8080/actuator/health || exit 1
 EXPOSE 8080
  5. 26.

    Building and running image # Build image from Dockerfile
 docker

    build -t my-todo-web-service:1.0.0 .
 
 # Run built image in container
 docker run -d -p 8080:8080 my-todo-web-service:1.0.0
  6. 27.

    Containerization with Google Jib Build images without the need for

    Docker Engine https://github.com/GoogleContainerTools/jib
  7. 32.

    Google Jib in Maven build <project>
 ...
 <build>
 <plugins>
 <plugin>


    <groupId>com.google.cloud.tools</groupId>
 <artifactId>jib-maven-plugin</artifactId>
 <version>1.6.0</version>
 <configuration>
 <to>
 <image>bmuschko/my-web-service:1.0.0</image>
 </to>
 </configuration>
 </plugin>
 </plugins>
 </build>
 </project> $ ./mvnw compile jib:dockerBuild
  8. 33.

    Google Jib in Gradle build plugins { id 'java' id

    'com.google.cloud.tools.jib' version ‘1.6.0' } jib.to.image = 'bmuschko/my-web-service:1.0.0' $ ./gradlew jibDockerBuild
  9. 34.
  10. 35.

    Pushing image # Log into Docker registry
 docker login --username=bmuschko↵

    —email=benjamin.muschko@gmail.com
 
 # Tag the image
 docker tag bb38976d03cf bmuschko/todo-web-service:1.0.0
 
 # Push image to Docker registry
 docker push bmuschko/todo-web-service
  11. 37.
  12. 44.

    Pull image and start/stop container # Pull image from registry

    (or build locally)
 docker pull bmuschko/todo-web-service:1.0.0
 
 # Start container
 docker run -d -p 8080:8080 -name todo-web-service↵ my-todo-web-service:1.0.0
 
 # Stop container
 docker container stop todo-web-service
 
 # Remove container
 docker container rm todo-web-service
  13. 51.

    Testcontainers in Gradle build repositories {
 mavenCentral()
 }
 
 dependencies

    {
 testImplementation 'org.testcontainers:junit-jupiter:1.12.0'
 testImplementation 'org.testcontainers:postgresql:1.12.0'
 testRuntime 'org.postgresql:postgresql'
 }
  14. 52.

    Integration test with JUnit 5 @DataJpaTest
 public class ToDoRepositoryIntegrationTest {


    @Autowired
 private ToDoRepository repository;
 
 @Test
 public void testCanSaveNewToDoItem() {
 ToDoItem toDoItem = createToDoItem("Buy milk");
 assertNull(toDoItem.getId());
 repository.save(toDoItem);
 assertNotNull(toDoItem.getId());
 }
 
 private ToDoItem createToDoItem(String name) {
 ToDoItem toDoItem = new ToDoItem();
 toDoItem.setName(name);
 return toDoItem;
 }
 } Class under test
  15. 53.

    Containerized database as fixture @Testcontainers
 @AutoConfigureTestDatabase(replace = NONE)
 @ContextConfiguration(initializers =

    { ToDoRepositoryIntegrationTest.Initializer.class })
 public class ToDoRepositoryIntegrationTest {
 @Container
 public static PostgreSQLContainer postgreSQLContainer = createDbContainer();
 
 private static PostgreSQLContainer createDbContainer() {
 return new PostgreSQLContainer("postgres:9.6.10-alpine")
 .withUsername("postgres")
 .withPassword("postgres")
 .withDatabaseName("todo");
 }
 
 static class Initializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
 @Override
 public void initialize(ConfigurableApplicationContext configurableApplicationContext) {
 TestPropertyValues.of(
 "spring.datasource.url=" + postgreSQLContainer.getJdbcUrl(),
 "spring.datasource.username=" + postgreSQLContainer.getUsername(),
 "spring.datasource.password=" + postgreSQLContainer.getPassword(),
 "spring.datasource.driver-class-name=org.postgresql.Driver",
 "spring.jpa.generate-ddl=true"
 ).applyTo(configurableApplicationContext.getEnvironment());
 }
 }
 } Create database container Inject database Configuration
  16. 54.

    Functional test with JUnit 5 public class ToDoWebServiceFunctionalTest {
 @Test


    @DisplayName("can retrieve all items before and after inserting new ones")
 void retrieveAllItems() {
 String allItems = getAllItems();
 assertEquals("[]", allItems);
 
 ToDoItem toDoItem = new ToDoItem();
 toDoItem.setName("Buy milk");
 toDoItem.setCompleted(false);
 insertItem(toDoItem);
 
 allItems = getAllItems();
 assertEquals("[{\"id\":1,\"name\":\"Buy milk\",\"completed\":false}]", allItems);
 } } Calls to HTTP endpoints
  17. 55.

    Containerized application as fixture @Testcontainers
 public class ToDoWebServiceFunctionalTest {
 @Container


    private static GenericContainer appContainer = createContainer();
 
 private static GenericContainer createContainer() {
 return new GenericContainer(buildImageDockerfile())
 .withExposedPorts(8080)
 .waitingFor(Wait.forHttp("/actuator/health")
 .forStatusCode(200));
 }
 
 private static ImageFromDockerfile buildImageDockerfile() {
 return new ImageFromDockerfile()
 .withFileFromFile(ARCHIVE_NAME, new File(DISTRIBUTION_DIR, ARCHIVE_NAME))
 .withDockerfileFromBuilder(builder -> builder
 .from("openjdk:jre-alpine")
 .copy(ARCHIVE_NAME, "/app/" + ARCHIVE_NAME)
 .entryPoint("java", "-jar", "/app/" + ARCHIVE_NAME)
 .build());
 }
 } Build container on-the-fly
  18. 56.
  19. 57.

    Running a multi-container app # Start composed apps
 docker-compose up


    
 # Stop composed apps
 docker-compose down
  20. 58.

    Compose test with JUnit 5 @ExtendWith(SpringExtension.class) @SpringBootTest
 public class ToDoAppIntegrationTest

    {
 @Autowired
 private ToDoService service;
 
 @Test
 public void canCreateNewItemAndRetrieveIt() {
 ToDoItem newItem = newItem("Buy milk");
 assertNull(newItem.getId());
 service.save(newItem);
 assertNotNull(newItem.getId());
 ToDoItem retrievedItem = service.findOne(newItem.getId());
 assertEquals(newItem, retrievedItem);
 } } Class under test
  21. 59.

    Compose fixture @Testcontainers
 public class ToDoAppImplIntegrationTest {
 private static final

    Logger logger = LoggerFactory.getLogger(ToDoAppImplIntegrationTest.class);
 private final static String WEB_SERVICE_NAME = "webservice_1";
 private final static int WEB_SERVICE_PORT = 8080;
 private final static String DATABASE_NAME = "database_1";
 private final static int DATABASE_PORT = 5432;
 
 @Container
 public static DockerComposeContainer environment = createComposeContainer();
 
 private static DockerComposeContainer createComposeContainer() {
 return new DockerComposeContainer(new File("src/integrationTest/resources/compose-test.yml"))
 .withLogConsumer(WEB_SERVICE_NAME, new Slf4jLogConsumer(logger))
 .withExposedService(WEB_SERVICE_NAME, WEB_SERVICE_PORT,
 Wait.forHttp("/actuator/health")
 .forStatusCode(200))
 .withExposedService(DATABASE_NAME, DATABASE_PORT,
 Wait.forLogMessage(".*database system is ready to accept connections.*\\s", 2));
 } } Create Compose container
  22. 60.

    Compose test file version: "2.0"
 services:
 database:
 image: "postgres:9.6.10-alpine"
 environment:


    - POSTGRES_USER=postgres
 - POSTGRES_PASSWORD=postgres
 - POSTGRES_DB=todo
 webservice:
 image: "bmuschko/todo-web-service:1.0.0"
 environment:
 - SPRING_PROFILES_ACTIVE=dev
 depends_on:
 - database
  23. 61.
  24. 66.

    Invoke build with Wrapper pipeline { stages { stage('Compile &

    Unit Tests') { steps { gradlew('clean', 'test') } } } ... } def gradlew(String... args) { sh "./gradlew ${args.join(' ')} -s" }
  25. 68.

    Injecting credentials & env vars stage('Push Image') { environment {

    DOCKER_USERNAME = "${env.DOCKER_USERNAME}" DOCKER_PASSWORD = credentials('DOCKER_PASSWORD') DOCKER_EMAIL = "${env.DOCKER_EMAIL}" } steps { gradlew('dockerPushImage') } }
  26. 71.

    Containerized pipeline stages pipeline {
 agent {
 kubernetes {
 defaultContainer

    'jnlp'
 yaml """
 apiVersion: v1
 kind: Pod
 metadata:
 labels:
 some-label: some-label-value
 spec:
 containers:
 - name: maven
 image: maven:alpine
 command:
 - cat
 tty: true
 - name: busybox
 image: busybox
 command:
 - cat
 tty: true
 """
 }
 }
 } stages {
 stage('Run in Maven container') {
 steps {
 container('maven') {
 sh 'mvn -version'
 }
 }
 }
 stage('Run in Busybox container') {
 steps {
 container('busybox') {
 sh '/bin/busybox'
 }
 }
 }
 stage('Run in default container') {
 steps {
 sh "echo 'Running in default container'"
 }
 }
 }
  27. 72.