Makefile, automating daily tasks & simplifying Gettings Started

This presentation dive inside Makefile workflow, properties and advanced features. With lots of examples of code it gives you everything you need to know to start using Makefiles as a tool for automating your daily tasks and also for simplifying users getting started with your project.

Transcript

1. Makefile
   Automating daily tasks
   & simplifying getting started
   Gustavo Pantuza
   

   View Slide

2. @gpantuza @pantuza
   https://blog.pantuza.com
   

   View Slide

3. Desconto de
   50%
   na compra de
   SSL
   Utilize o código
   DEVBR50OFF
   

   View Slide

4. Agenda
   Introduction Properties Advanced
   ● Motivation
   ● How it works
   ● File Format
   ● Demo
   ● Variables
   ● Conditionals
   ● Builtin functions
   ● Includes
   ● Defining Functions
   ● Automatic Variables
   ● Caveats
   ● Other directory
   ● Use cases
   

   View Slide

5. Introduction
   

   View Slide

6. ● Automation
   ● Simplicity
   ● Learning curve
   ● Getting Started
   ● Almost no dependencies
   Motivation
   

   View Slide

7. ● Program make
   zypper install make
   apk add make
   apt-get install make
   ● A Makefile
   The file which make reads in
   order to execute rules
   How it work
   $> make build
   

   View Slide

8. By the original definition make is:
   “The make utility
   automatically determines
   which pieces of a large
   program need to be
   recompiled, and issues
   commands to recompile them.”
   How it work
   $> make what
   Overview
   

   View Slide

9. $> make build
   Building docker image
   Sending build context to Docker daemon 73.73kB
   Step 1/4 : FROM golang:1.14-alpine
   ---> 30df784d6206
   Step 2/4 : COPY src /app
   ---> 5fa423e5e748
   Step 3/4 : WORKDIR /app
   ---> Running in 3a969fe98be8
   Removing intermediate container 3a969fe98be8
   ---> 3a59a136b09c
   Step 4/4 : CMD go run main.go
   ---> Running in 49907a2f877a
   Removing intermediate container 49907a2f877a
   ---> 14ec4f19142c
   Successfully built 14ec4f19142c
   Successfully tagged makefile:latest
   How it work
   $> make build
   Overview
   

   View Slide

10. make rules
    File format
    Rule Shape
    

    View Slide

11. target: dependency
    command
    File format
    Rule Shape
    

    View Slide

12. ● target
    Action to be executed by the rule
    ● dependencies
    Needed to exist to satisfy the target
    ● commands
    Sequence of commands to be executed
    File format
    

    View Slide

13. target..: dependency..
    command..
    ..
    File format
    Rule Shape
    

    View Slide

14. target..: dependency..
    command..
    ..
    File format
    Rule Shape
    

    View Slide

15. Hands On
    

    View Slide

16. Properties
    

    View Slide

17. # Branch that triggers production deployment
    PROD_BRANCH := production
    # Push current branch changes to production branch
    push:
    $(info Pushing changes to $(PROD_BRANCH))
    @git push origin $(PROD_BRANCH)
    Variables
    $> make push
    Pushing changes to production
    Everything up-to-date
    

    View Slide

18. # Docker image version
    IMG_VERSION ?= latest
    # Builds the docker image based on the Dockerfile
    build: Dockerfile
    $(info Building docker image)
    @docker build --tag makefile:$(IMG_VERSION) .
    Variables
    $> make build
    ...
    Successfully tagged makefile:latest
    $> make build IMG_VERSION=0.2
    ...
    Successfully tagged makefile:0.2
    $> make build IMG_VERSION=beta
    ...
    Successfully tagged makefile:beta
    

    View Slide

19. # Source code directory
    SRC := src
    GO_FILES := $(SRC)/main.go $(SRC)/main_test.go
    # Runs code format on Golang files
    fmt: $(GO_FILES)
    $(info Running Go fmt on $(GO_FILES))
    @go fmt $^
    Variables
    $> make fmt
    Running Go fmt on src/main.go src/main_test.go
    

    View Slide

20. # Expanded once
    VERSION := 0.12
    # Expanded on call
    NOW = $(shell date +%s)
    # Conditional assignment
    IMG_TAG ?= latest
    # Appending values
    CFLAGS += -O3
    Variables
    

    View Slide

21. # Rolls back application to specific version
    rollback:
    ifndef ROLLBACK_VERSION
    $(error Missing ROLLBACK_VERSION variable)
    endif
    @kubectl rollout undo deployment app --to-revision=$(ROLLBACK_VERSION)
    Conditionals
    $> make rollback
    Makefile:36: *** Missing ROLLBACK_VERSION variable. Stop.
    $> make rollback ROLLBACK_VERSION=15
    ...
    

    View Slide

22. # Operating system name
    OS := $(shell uname)
    # Install project dependencies on the Operating System
    deps:
    $(info Installing project dependencies locally)
    ifeq ($(OS),Linux)
    @zypper install --no-confirm nginx
    else ifeq ($(OS), Darwin)
    @brew install nginx
    else
    $(warning Unsupported Operating System)
    endif
    Conditionals
    $> make deps # Else example
    Makefile:36: Unsupported Operating System
    $> make deps # Linux example
    Installing project dependencies locally
    ...
    Conditionals
    

    View Slide

23. # List of files to be modified
    FILES := src/a src/b bin/a bin/b
    # String functions examples
    functions:
    @echo "Input: $(FILES)"
    @echo "subst -> " $(subst src, tmp, $(FILES))
    @echo "filter -> " $(filter bin%, $(FILES))
    @echo "findstring -> " $(findstring a, $(FILES))
    @echo "sort -> " $(sort $(FILES))
    @echo "word -> " $(word 2, $(FILES))
    @echo "lastword -> " $(lastword, $(FILES))
    Builtin functions
    $> make functions
    Input: src/a src/b bin/a bin/b
    subst -> tmp/a tmp/b bin/a bin/b
    filter -> bin/a bin/b
    findstring -> a
    sort -> bin/a bin/b src/a src/b
    word -> src/b
    lastword -> bin/b
    Text Functions
    

    View Slide

24. # List of files to be modified
    FILES := src/a src/b bin/a bin/b
    # Files functions examples
    functions:
    @echo "Input: $(FILES)"
    @echo "dir -> " $(dir $(FILES))
    @echo "notdir -> " $(notdir $(FILES))
    @echo "basename -> " $(basename $(FILES))
    @echo "addsuffix -> " $(addsuffix .go, $(FILES))
    @echo "addprefix -> " $(addprefix /tmp/, $(FILES))
    @echo "abspath -> " $(abspath $(FILES))
    Builtin functions
    $> make functions
    Input: src/a src/b bin/a bin/b
    dir -> src/ src/ bin/ bin/
    notdir -> a b a b
    basename -> src/a src/b bin/a bin/b
    addsuffix -> src/a.go src/b.go bin/a.go bin/b.go
    addprefix -> /tmp/src/a /tmp/src/b /tmp/bin/a /tmp/bin/b
    abspath -> /makefile/src/a /makefile/src/b /makefile/bin/a /makefile/bin/b Files Functions
    

    View Slide

25. # List of directories
    DIRS := src bin logs
    # Directories with suffix
    SUF_DIRS := $(foreach dir, $(DIRS), $(addsuffix -dir, $(dir)))
    # Runtime functions examples
    runtime:
    @echo "$(shell hostname)"
    @echo $(SUF_DIRS)
    @echo $(wildcard src/*_test.go)
    $(eval AWS_ACCOUNT = $(shell aws sts get-caller-identity --query "Account"))
    @echo $(AWS_ACCOUNT)
    Builtin functions
    $> make runtime
    opensuse-machine
    src-dir bin-dir logs-dir
    src/main_test.go
    151666326780
    

    View Slide

26. # Builtin functions to print messages
    messages:
    $(info Informative function)
    $(warning Warning function)
    $(error Error function)
    Builtin functions
    $> make messages
    Informative function
    Makefile:40: Warning function
    Makefile:41: *** Error function. Stop.
    Text Functions
    

    View Slide

27. # Includes an external Makefile
    include Makefile.conf
    # Gets project version read
    # included Makefile.conf
    version:
    @echo $(VERSION)
    @echo $(PROJECT_NAME)
    @echo $(DOCS)
    @echo $(PKG_MGR)
    Includes
    $> make version
    0.42
    makefile
    docs
    /usr/bin/zypper
    # =====* Makefile.conf *=====
    # Shell program
    SHELL := $(shell which bash)
    # Gets from shell the operating system
    OS := $(shell uname -s)
    PKG_MGR := $(shell which zypper)
    ifeq ($(OS), Darwin)
    PKG_MGR = $(shell which brew)
    endif
    # Project name
    PROJECT_NAME := makefile
    # Get the version from file
    VERSION := 0.42
    # Documentation directory
    DOCS := docs
    

    View Slide

28. Advanced
    

    View Slide

29. #
    # Returns current timestamp (no parameters)
    #
    NOW = $(shell date +%s)
    #
    # Function to subtract the first number from the second
    # (with parameters)
    SUBTRACT = $(shell expr $(1) - $(2) )
    time:
    @echo "Now: $(call NOW)"
    @echo "Subtract 20 from 30: $(call SUBTRACT, 30, 20)"
    Functions
    $> make time
    Now: 1595898056
    Subtract 20 from 30: 10
    

    View Slide

30. #
    # Function that takes a template and a variable as input.
    # It returns template text interpolated with the variable.
    #
    define parse_template
    @echo $(1) | sed 's/{NAME}/$(2)/'
    endef
    # Builds templates dynamically
    template:
    $(call parse_template,"Learning {NAME} rocks!",Makefile)
    $(call parse_template,"Learning {NAME} rocks!",Computer Science)
    $(call parse_template,"Learning {NAME} rocks!",DevOps)
    Functions
    $> make template
    Learning Makefile rocks!
    Learning Computer Science rocks!
    Learning DevOps rocks!
    

    View Slide

31. # Magic variables example
    magic: Dockerfile README.md LICENSE
    @echo $<
    @echo $^
    @echo $@
    Automatic variables
    $> make magic
    Dockerfile
    Dockerfile README.md LICENSE
    magic
    Automatic Variables
    

    View Slide

32. # Resolve git command path
    GIT := $(shell which git)
    push:
    @$(GIT) push origin master
    Caveats
    $> make push
    Enumerating objects: 5, done.
    Counting objects: 100% (5/5), done.
    Delta compression using up to 8 threads
    Compressing objects: 100% (3/3), done.
    Writing objects: 100% (3/3), 1014 bytes | 1014.00 KiB/s, done.
    Total 3 (delta 2), reused 0 (delta 0)
    remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
    To github.com:pantuza/makefile.git
    d865102..493d7cd master -> master
    

    View Slide

33. # @ before command avoid printing the command
    hide:
    @echo "No command shown, only result"
    show:
    echo "Command shown and also the result"
    Caveats
    $> make hide
    No command shown, only result
    $> make show
    echo "Command shown and also the result"
    Command shown and also the result
    

    View Slide

34. # .PHONY make sure to not fail if the target is
    # equal to a file or directory
    .PHONY: src
    src:
    @echo "Hi PHONY target"
    Caveats
    $> ls
    Dockerfile LICENSE Makefile README.md src
    $> make src # Whitout .PHONY
    make: 'src' is up to date.
    $> make src # Using .PHONY
    Hi PHONY target
    

    View Slide

35. # Using Wildcards
    # Validates all OpenAPI spec files
    check_spec: $(SRC)/%.yaml
    @openapi-spec-validator $^
    Caveats
    $> make check_spec
    OK
    

    View Slide

36. Caveats
    # Chaining rules
    $> make start logs
    Running application container
    6de3d88c680d57da5658a5007b988cdaed909e8cb588af20b1bc9100ca2bad07
    [Http] Status=Listening, Port=8080
    

    View Slide

37. Caveats
    # Debugging with -n option
    $> make build start logs -n
    Building docker image
    docker build --tag makefile:latest .
    Running application container
    docker run -it --rm -p 8080:8080 --name makefile -d
    makefile:latest
    docker logs --tail 50 --follow makefile
    

    View Slide

38. # Makefile on an inner directory
    # For example /lib/Makefile
    binary: main.go
    @go build -o $@
    Other directories
    $> make -C lib/ binary
    make: Entering directory '/makefile/lib'
    make: Leaving directory '/makefile/lib'
    

    View Slide

39. Use cases
    # Jenkinsfile
    ...
    stage('Unit Tests') {
    steps {
    echo 'Running Unit Tests..'
    sh 'make unit_test'
    }
    }
    stage('Integration Tests') {
    steps {
    echo 'Running Integration Tests..'
    sh 'make integration_test'
    }
    }
    ...
    

    View Slide

40. Use cases
    # .travis.yml
    sudo: required
    language: cpp
    services:
    - docker
    before_install:
    - make start
    script:
    - make test
    

    View Slide

41. Use cases
    # DRY - Don’t Repeat Yourself
    # CI/CD Pipeline execution chain
    pipeline: pep8 check_openapi check_k8s unit_test integration_test acceptance_test
    # Deploy application to production
    deploy: cloud_auth build_img push_img migrate_db patch
    @kubectl rollout status deployment $(DEPLOYMENT_NAME)
    @kubectl apply -f $(K8S_DIR)/deployment.yaml
    

    View Slide

42. Use cases
    # Getting started
    In order to have the project running locally
    you can just run as follows:
    ```bash
    $> make setup
    ```
    > It will install all project dependencies and
    > and run application using containers
    You can check if everything is OK by running
    the tests:
    ```bash
    $> make test
    ```
    If everything went well, please refer to the
    CONTRIBUTING.md and start contributing to the
    Project!
    

    View Slide

43. Use cases
    Kubernetes
    Python language
    NodeJS
    BusyBox
    Docker CLI
    Prometheus
    Traefik
    Terraform
    

    View Slide

44. Automating & simplifying
    

    View Slide

45. https://github.com/pantuza/makefile
    https://www.gnu.org/software/make/manual/make.html
    

    View Slide

46. Desconto de
    50%
    na compra de
    SSL
    Utilize o código
    DEVBR50OFF
    

    View Slide

47. https://blog.pantuza.com
    https://github.com/pantuza
    https://twitter.com/gpantuza
    Dúvidas?
    

    View Slide