Make for Microservices

http://www.devfestdc.org/sessions/using-makemakefile-for-microservices-developer-workflow-and-cicd/

Developing and delivering microservices can be hard. A given service may depend on many other services at runtime, making it tough to get a tight feedback loop during development. Tools like docker-compose help, but they still leave tasks like database bootstrapping and service initialization as manual work for the user. And what about REPL-driven development, hot code reloading, attaching debuggers, and projects that include several languages and build systems? Similar challenges arise when it comes time to integrate, test, and deploy the service.

What if there was a simple, standard tool available on ~all *nix operating systems that could be used to correctly sequence and orchestrate inter-dependent development, build, and deployment tasks? There is, and it’s been available since 1976: Make!

In this talk, we’ll explore using Make as a meta-build/deploy system to solve the problems described above during the development workflow and within CI/CD pipelines.

Transcript

1. Make for Microservices
   Useful conventions for developer workflow and CI/CD
   

   View Slide

2. Hi, I’m Bobby
   I’m on the Technology Fellow’s team at
   I dislike accidental complexity
   bobby.calderwood@capitalone.com
   @bobbycalderwood
   https://github.com/bobby
   

   View Slide

3. The Problem
   • Encountered on several recent projects, including https://
   tinyurl.com/capital-one-cmdr
   • How to interactively develop one or more services which
   require several other services at runtime?
   • How to test such an application locally?
   • How to build, deploy services which use a variety of
   different build and deploy toolchains?
   • How can developers joining the project make sense of any
   of the above?
   

   View Slide

4. Partial Solutions
   Pro Con
   Language build tools
   Run tests, manage library
   dependencies, build
   executable artifacts from
   my code
   Prolific, language-specific,
   plugins rather than .sh for
   extension
   Docker
   Consistent deployment
   artifact and runtime target
   Lang build → docker build,
   adds another layer, poor
   dev affordances
   Docker Compose
   dev-time service
   orchestration, chaos
   testing
   Bootstrapping support
   services prior to app start,
   provision/deploy to prod
   Pile of scripts
   Cross-platform/cross-
   language: .sh is geek
   lingua-franca
   deps mgmt, no shared
   conventions, imperative/
   verbose README
   

   View Slide

5. A Better Solution
   • Make!
   • It’s been around since 1976
   • Many developers familiar with it, projects using it
   • Speaks .sh, so everyone can use/understand it
   • Acts as a meta-build tool to orchestrate the
   aforementioned
   

   View Slide

6. Example Project
   • 3 µ-services
   • write_service (Clojure)
   • stream_processor (Java)
   • read_service (Java)
   • 3 supporting services
   • Kafka
   • Zookeeper (only used by Kafka)
   • PostgreSQL
   

   View Slide

7. version: "2"
   services:
   write_service:
   build: ./clojure-leiningen-rest-service
   image: "419056267649.dkr.ecr.us-east-1.amazonaws.com/write_service"
   ports:
   - "3000:3000"
   stream_processor:
   build: ./java-dropwizard-rest-service
   image: "419056267649.dkr.ecr.us-east-1.amazonaws.com/stream_processor"
   ports:
   - "5005:5005"
   environment:
   JAVA_OPTS: "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
   read_service:
   build: ./java-dropwizard-rest-service
   image: "419056267649.dkr.ecr.us-east-1.amazonaws.com/read_service"
   ports:
   - "8080:8080"
   - "5006:5005"
   environment:
   JAVA_OPTS: "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
   postgres:
   image: postgres:9.6-alpine
   ports:
   - "5432:5432"
   zookeeper:
   image: wurstmeister/zookeeper:3.4.6
   kafka:
   image: wurstmeister/kafka:0.10.2.0
   environment:
   KAFKA_ADVERTISED_PORT: 9092
   KAFKA_ADVERTISED_HOST_NAME: kafka
   KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
   ports:
   - "9092:9092"
   

   View Slide

8. Dockerfile (Java services)
   FROM java:8-alpine
   MAINTAINER Bobby Calderwood
   ENV JAVA_OPTS=""
   RUN mkdir -p /opt
   ADD target/java-dropwizard-rest-service-*.jar /opt/app.jar
   EXPOSE 8080
   CMD java ${JAVA_OPTS} -jar /opt/app.jar
   

   View Slide

9. Dockerfile (Clojure service)
   FROM java:8-alpine
   MAINTAINER Bobby Calderwood
   RUN mkdir -p /opt
   ADD target/clojure-leiningen-rest-service-*-standalone.jar /opt/app.jar
   EXPOSE 8080
   CMD ["java", "-jar", "/opt/app.jar"]
   

   View Slide

10. Workflow Targets
    • Language artifact targets e.g. example/target/*.jar
    • check
    • run
    • stop
    • distclean
    • logs
    • Service interaction e.g. psql, kafka-console-consumer
    

    View Slide

11. # Clojure project build via Leiningen
    clojure-leiningen-rest-service/target/clojure-leiningen-rest-service-*-standalone.jar:
    cd clojure-leiningen-rest-service && lein uberjar
    # Java project build via Maven
    java-dropwizard-rest-service/target/java-dropwizard-rest-service-*.jar:
    cd java-dropwizard-rest-service && mvn package
    # Java project build via Maven
    java-dropwizard-stream-processor/target/java-dropwizard-stream-processor-*.jar:
    cd java-dropwizard-stream-processor && mvn package
    # Run all tests
    .PHONY: check
    check:
    cd clojure-leiningen-rest-service && lein test
    cd java-dropwizard-rest-service && mvn test
    cd java-dropwizard-stream-processor && mvn test
    

    View Slide

12. # Run, bootstrap, migrate, and then display status of all services via docker-
    compose
    .PHONY: run
    run: database-migrate # TEMPORARY HAND-WAVING!!!
    docker-compose --project-name=makefile_microservices ps
    # Stop the services (but preserve the images and state)
    .PHONY: stop
    stop:
    docker-compose --project-name=makefile_microservices stop
    # Reset all docker/docker-compose local state
    .PHONY: distclean
    distclean: clean
    -docker-compose --project-name=makefile_microservices rm -f -v
    -docker network rm makefile_microservices
    

    View Slide

13. # Tail the docker-compose logs
    .PHONY: logs
    logs:
    docker-compose --project-name=makefile_microservices logs $(LOG_OPTIONS)
    # Run psql in a docker container, connected to the postgres service
    .PHONY: psql
    psql:
    docker run --network makefile_microservices --rm -it --entrypoint psql $
    (POSTGRES_IMAGE) -h postgres -U postgres -d $(DATABASE_NAME) $(args)
    # Tail KAFKA_TOPIC ("test" by default) via kafka-console-consumer in a docker
    container
    .PHONY: kafka-console-consumer
    kafka-console-consumer:
    docker run --network makefile_microservices --rm -it --entrypoint /opt/kafka/
    bin/kafka-console-consumer.sh $(KAFKA_IMAGE) --bootstrap-server kafka:9092 --
    topic $(KAFKA_TOPIC) $(args)
    

    View Slide

14. Utility/Orchestration Targets
    • _network
    • _run
    • _sleep
    • database-bootstrap, database-migrate
    

    View Slide

15. # Create docker external network used by non-compose-managed containers to connect
    to compose-managed services
    .PHONY: _network
    _network:
    -docker network create makefile_microservices
    # Run all services via docker-compose
    .PHONY: _run
    _run: _network
    docker-compose --project-name=makefile_microservices up -d
    # Run all services via docker-compose
    .PHONY: _sleep
    _sleep:
    sleep $(SLEEP_SECONDS)
    # Create postgres database via psql
    .PHONY: database-bootstrap
    database-bootstrap: all _run _sleep
    -docker run --network makefile_microservices --rm -it --entrypoint createdb $
    (POSTGRES_IMAGE) -h postgres -U postgres $(DATABASE_NAME)
    # Migrate postgres database via psql
    .PHONY: database-migrate
    database-migrate: database-bootstrap
    -docker run --network makefile_microservices --rm -i --entrypoint psql $
    (POSTGRES_IMAGE) -h postgres -U postgres -d $(DATABASE_NAME) < $
    (DATABASE_MIGRATION_PATH)
    

    View Slide

16. REPL-driven Development
    # Stops the write_service, and launches a Clojure REPL instead
    .PHONY: repl
    repl: run
    docker-compose stop write_service
    cd clojure-leiningen-rest-service && lein repl :headless
    

    View Slide

17. Attaching Debuggers
    read_service:
    build: ./java-dropwizard-rest-service
    image: "419056267649.dkr.ecr.us-east-1.amazonaws.com/read_service"
    ports:
    - "8080:8080"
    - "5005:5005"
    environment:
    JAVA_OPTS: "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
    

    View Slide

18. Workflow Demo
    

    View Slide

19. CI/CD Targets
    • all (default)
    • clean
    • provision
    • push
    • deploy
    

    View Slide

20. default: all
    # Build all docker images
    .PHONY: all
    all: clojure-leiningen-rest-service/target/clojure-leiningen-rest-service-*-standalone.jar
    java-dropwizard-rest-service/target/java-dropwizard-rest-service-*.jar java-dropwizard-
    stream-processor/target/java-dropwizard-stream-processor-*.jar
    docker-compose --project-name=makefile_microservices build
    # Delete all built artifacts, both language builds and docker images
    .PHONY: clean
    clean:
    -docker-compose --project-name=makefile_microservices down --rmi local
    -cd clojure-leiningen-rest-service && lein clean
    -cd java-dropwizard-rest-service && mvn clean
    -cd java-dropwizard-stream-processor && mvn clean
    # Build a Kubernetes cluster via kops
    .PHONY: provision
    provision:
    kops create cluster --cloud=aws --node-count=3 --node-size=t2.small --zones $(AWS_ZONES)
    --master-size=t2.large --master-zones $(AWS_ZONES) --name $(KOPS_CLUSTER_NAME) --state $
    (KOPS_STATE_STORE) --ssh-public-key=~/.ssh/id_rsa.pub --yes
    # Push Docker images to ECR
    .PHONY: push
    push: all
    docker-compose --project-name=makefile_microservices push
    # Deploy the services to Kubernetes
    .PHONY: deploy
    deploy: push
    kompose up
    

    View Slide

21. CI/CD Demo
    

    View Slide

22. Opportunities for
    Improvement
    • Too much .PHONY, creates unnecessary re-work
    • Could possibly track state of environment via local
    files?
    • Makefile, docker-compose.yml, and service builds are
    somewhat coupled
    • Top-level file doing a lot, not using Make’s advanced
    features for loading service-specific rules/targets
    • Use Docker containers for kops and kompose calls
    

    View Slide

23. Benefits
    • Consistent, unsurprising developer affordances
    • Declarative documentation, invocation
    • Imperative implementation
    • Consistent, unsurprising CI/CD integration points
    • Feel grayer around the temples…
    

    View Slide

24. Questions?
    

    View Slide

25. References
    • https://www.gnu.org/software/make/manual/
    • http://david.rothlis.net/large-gnu-make/
    • http://sites.e-advies.nl/nonrecursive-make.html
    • https://github.com/bobby/makefile_microservices
    

    View Slide