Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Makefile, automating daily tasks & simplifying Gettings Started

Makefile, automating daily tasks & simplifying Gettings Started

This presentation dive inside Makefile workflow, properties and advanced features. With lots of examples of code it gives you everything you need to know to start using Makefiles as a tool for automating your daily tasks and also for simplifying users getting started with your project.

Gustavo Pantuza

July 28, 2020
Tweet

More Decks by Gustavo Pantuza

Other Decks in Programming

Transcript

  1. Makefile
    Automating daily tasks
    & simplifying getting started
    Gustavo Pantuza

    View Slide

  2. @gpantuza @pantuza
    https://blog.pantuza.com

    View Slide

  3. Desconto de
    50%
    na compra de
    SSL
    Utilize o código
    DEVBR50OFF

    View Slide

  4. Agenda
    Introduction Properties Advanced
    ● Motivation
    ● How it works
    ● File Format
    ● Demo
    ● Variables
    ● Conditionals
    ● Builtin functions
    ● Includes
    ● Defining Functions
    ● Automatic Variables
    ● Caveats
    ● Other directory
    ● Use cases

    View Slide

  5. Introduction

    View Slide

  6. ● Automation
    ● Simplicity
    ● Learning curve
    ● Getting Started
    ● Almost no dependencies
    Motivation

    View Slide

  7. ● Program make
    zypper install make
    apk add make
    apt-get install make
    ● A Makefile
    The file which make reads in
    order to execute rules
    How it work
    $> make build

    View Slide

  8. By the original definition make is:
    “The make utility
    automatically determines
    which pieces of a large
    program need to be
    recompiled, and issues
    commands to recompile them.”
    How it work
    $> make what
    Overview

    View Slide

  9. $> make build
    Building docker image
    Sending build context to Docker daemon 73.73kB
    Step 1/4 : FROM golang:1.14-alpine
    ---> 30df784d6206
    Step 2/4 : COPY src /app
    ---> 5fa423e5e748
    Step 3/4 : WORKDIR /app
    ---> Running in 3a969fe98be8
    Removing intermediate container 3a969fe98be8
    ---> 3a59a136b09c
    Step 4/4 : CMD go run main.go
    ---> Running in 49907a2f877a
    Removing intermediate container 49907a2f877a
    ---> 14ec4f19142c
    Successfully built 14ec4f19142c
    Successfully tagged makefile:latest
    How it work
    $> make build
    Overview

    View Slide

  10. make rules
    File format
    Rule Shape

    View Slide

  11. target: dependency
    command
    File format
    Rule Shape

    View Slide

  12. ● target
    Action to be executed by the rule
    ● dependencies
    Needed to exist to satisfy the target
    ● commands
    Sequence of commands to be executed
    File format

    View Slide

  13. target..: dependency..
    command..
    ..
    File format
    Rule Shape

    View Slide

  14. target..: dependency..
    command..
    ..
    File format
    Rule Shape

    View Slide

  15. Hands On

    View Slide

  16. Properties

    View Slide

  17. # Branch that triggers production deployment
    PROD_BRANCH := production
    # Push current branch changes to production branch
    push:
    $(info Pushing changes to $(PROD_BRANCH))
    @git push origin $(PROD_BRANCH)
    Variables
    $> make push
    Pushing changes to production
    Everything up-to-date

    View Slide

  18. # Docker image version
    IMG_VERSION ?= latest
    # Builds the docker image based on the Dockerfile
    build: Dockerfile
    $(info Building docker image)
    @docker build --tag makefile:$(IMG_VERSION) .
    Variables
    $> make build
    ...
    Successfully tagged makefile:latest
    $> make build IMG_VERSION=0.2
    ...
    Successfully tagged makefile:0.2
    $> make build IMG_VERSION=beta
    ...
    Successfully tagged makefile:beta

    View Slide

  19. # Source code directory
    SRC := src
    GO_FILES := $(SRC)/main.go $(SRC)/main_test.go
    # Runs code format on Golang files
    fmt: $(GO_FILES)
    $(info Running Go fmt on $(GO_FILES))
    @go fmt $^
    Variables
    $> make fmt
    Running Go fmt on src/main.go src/main_test.go

    View Slide

  20. # Expanded once
    VERSION := 0.12
    # Expanded on call
    NOW = $(shell date +%s)
    # Conditional assignment
    IMG_TAG ?= latest
    # Appending values
    CFLAGS += -O3
    Variables

    View Slide

  21. # Rolls back application to specific version
    rollback:
    ifndef ROLLBACK_VERSION
    $(error Missing ROLLBACK_VERSION variable)
    endif
    @kubectl rollout undo deployment app --to-revision=$(ROLLBACK_VERSION)
    Conditionals
    $> make rollback
    Makefile:36: *** Missing ROLLBACK_VERSION variable. Stop.
    $> make rollback ROLLBACK_VERSION=15
    ...

    View Slide

  22. # Operating system name
    OS := $(shell uname)
    # Install project dependencies on the Operating System
    deps:
    $(info Installing project dependencies locally)
    ifeq ($(OS),Linux)
    @zypper install --no-confirm nginx
    else ifeq ($(OS), Darwin)
    @brew install nginx
    else
    $(warning Unsupported Operating System)
    endif
    Conditionals
    $> make deps # Else example
    Makefile:36: Unsupported Operating System
    $> make deps # Linux example
    Installing project dependencies locally
    ...
    Conditionals

    View Slide

  23. # List of files to be modified
    FILES := src/a src/b bin/a bin/b
    # String functions examples
    functions:
    @echo "Input: $(FILES)"
    @echo "subst -> " $(subst src, tmp, $(FILES))
    @echo "filter -> " $(filter bin%, $(FILES))
    @echo "findstring -> " $(findstring a, $(FILES))
    @echo "sort -> " $(sort $(FILES))
    @echo "word -> " $(word 2, $(FILES))
    @echo "lastword -> " $(lastword, $(FILES))
    Builtin functions
    $> make functions
    Input: src/a src/b bin/a bin/b
    subst -> tmp/a tmp/b bin/a bin/b
    filter -> bin/a bin/b
    findstring -> a
    sort -> bin/a bin/b src/a src/b
    word -> src/b
    lastword -> bin/b
    Text Functions

    View Slide

  24. # List of files to be modified
    FILES := src/a src/b bin/a bin/b
    # Files functions examples
    functions:
    @echo "Input: $(FILES)"
    @echo "dir -> " $(dir $(FILES))
    @echo "notdir -> " $(notdir $(FILES))
    @echo "basename -> " $(basename $(FILES))
    @echo "addsuffix -> " $(addsuffix .go, $(FILES))
    @echo "addprefix -> " $(addprefix /tmp/, $(FILES))
    @echo "abspath -> " $(abspath $(FILES))
    Builtin functions
    $> make functions
    Input: src/a src/b bin/a bin/b
    dir -> src/ src/ bin/ bin/
    notdir -> a b a b
    basename -> src/a src/b bin/a bin/b
    addsuffix -> src/a.go src/b.go bin/a.go bin/b.go
    addprefix -> /tmp/src/a /tmp/src/b /tmp/bin/a /tmp/bin/b
    abspath -> /makefile/src/a /makefile/src/b /makefile/bin/a /makefile/bin/b Files Functions

    View Slide

  25. # List of directories
    DIRS := src bin logs
    # Directories with suffix
    SUF_DIRS := $(foreach dir, $(DIRS), $(addsuffix -dir, $(dir)))
    # Runtime functions examples
    runtime:
    @echo "$(shell hostname)"
    @echo $(SUF_DIRS)
    @echo $(wildcard src/*_test.go)
    $(eval AWS_ACCOUNT = $(shell aws sts get-caller-identity --query "Account"))
    @echo $(AWS_ACCOUNT)
    Builtin functions
    $> make runtime
    opensuse-machine
    src-dir bin-dir logs-dir
    src/main_test.go
    151666326780

    View Slide

  26. # Builtin functions to print messages
    messages:
    $(info Informative function)
    $(warning Warning function)
    $(error Error function)
    Builtin functions
    $> make messages
    Informative function
    Makefile:40: Warning function
    Makefile:41: *** Error function. Stop.
    Text Functions

    View Slide

  27. # Includes an external Makefile
    include Makefile.conf
    # Gets project version read
    # included Makefile.conf
    version:
    @echo $(VERSION)
    @echo $(PROJECT_NAME)
    @echo $(DOCS)
    @echo $(PKG_MGR)
    Includes
    $> make version
    0.42
    makefile
    docs
    /usr/bin/zypper
    # =====* Makefile.conf *=====
    # Shell program
    SHELL := $(shell which bash)
    # Gets from shell the operating system
    OS := $(shell uname -s)
    PKG_MGR := $(shell which zypper)
    ifeq ($(OS), Darwin)
    PKG_MGR = $(shell which brew)
    endif
    # Project name
    PROJECT_NAME := makefile
    # Get the version from file
    VERSION := 0.42
    # Documentation directory
    DOCS := docs

    View Slide

  28. Advanced

    View Slide

  29. #
    # Returns current timestamp (no parameters)
    #
    NOW = $(shell date +%s)
    #
    # Function to subtract the first number from the second
    # (with parameters)
    SUBTRACT = $(shell expr $(1) - $(2) )
    time:
    @echo "Now: $(call NOW)"
    @echo "Subtract 20 from 30: $(call SUBTRACT, 30, 20)"
    Functions
    $> make time
    Now: 1595898056
    Subtract 20 from 30: 10

    View Slide

  30. #
    # Function that takes a template and a variable as input.
    # It returns template text interpolated with the variable.
    #
    define parse_template
    @echo $(1) | sed 's/{NAME}/$(2)/'
    endef
    # Builds templates dynamically
    template:
    $(call parse_template,"Learning {NAME} rocks!",Makefile)
    $(call parse_template,"Learning {NAME} rocks!",Computer Science)
    $(call parse_template,"Learning {NAME} rocks!",DevOps)
    Functions
    $> make template
    Learning Makefile rocks!
    Learning Computer Science rocks!
    Learning DevOps rocks!

    View Slide

  31. # Magic variables example
    magic: Dockerfile README.md LICENSE
    @echo $<
    @echo $^
    @echo $@
    Automatic variables
    $> make magic
    Dockerfile
    Dockerfile README.md LICENSE
    magic
    Automatic Variables

    View Slide

  32. # Resolve git command path
    GIT := $(shell which git)
    push:
    @$(GIT) push origin master
    Caveats
    $> make push
    Enumerating objects: 5, done.
    Counting objects: 100% (5/5), done.
    Delta compression using up to 8 threads
    Compressing objects: 100% (3/3), done.
    Writing objects: 100% (3/3), 1014 bytes | 1014.00 KiB/s, done.
    Total 3 (delta 2), reused 0 (delta 0)
    remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
    To github.com:pantuza/makefile.git
    d865102..493d7cd master -> master

    View Slide

  33. # @ before command avoid printing the command
    hide:
    @echo "No command shown, only result"
    show:
    echo "Command shown and also the result"
    Caveats
    $> make hide
    No command shown, only result
    $> make show
    echo "Command shown and also the result"
    Command shown and also the result

    View Slide

  34. # .PHONY make sure to not fail if the target is
    # equal to a file or directory
    .PHONY: src
    src:
    @echo "Hi PHONY target"
    Caveats
    $> ls
    Dockerfile LICENSE Makefile README.md src
    $> make src # Whitout .PHONY
    make: 'src' is up to date.
    $> make src # Using .PHONY
    Hi PHONY target

    View Slide

  35. # Using Wildcards
    # Validates all OpenAPI spec files
    check_spec: $(SRC)/%.yaml
    @openapi-spec-validator $^
    Caveats
    $> make check_spec
    OK

    View Slide

  36. Caveats
    # Chaining rules
    $> make start logs
    Running application container
    6de3d88c680d57da5658a5007b988cdaed909e8cb588af20b1bc9100ca2bad07
    [Http] Status=Listening, Port=8080

    View Slide

  37. Caveats
    # Debugging with -n option
    $> make build start logs -n
    Building docker image
    docker build --tag makefile:latest .
    Running application container
    docker run -it --rm -p 8080:8080 --name makefile -d
    makefile:latest
    docker logs --tail 50 --follow makefile

    View Slide

  38. # Makefile on an inner directory
    # For example /lib/Makefile
    binary: main.go
    @go build -o $@
    Other directories
    $> make -C lib/ binary
    make: Entering directory '/makefile/lib'
    make: Leaving directory '/makefile/lib'

    View Slide

  39. Use cases
    # Jenkinsfile
    ...
    stage('Unit Tests') {
    steps {
    echo 'Running Unit Tests..'
    sh 'make unit_test'
    }
    }
    stage('Integration Tests') {
    steps {
    echo 'Running Integration Tests..'
    sh 'make integration_test'
    }
    }
    ...

    View Slide

  40. Use cases
    # .travis.yml
    sudo: required
    language: cpp
    services:
    - docker
    before_install:
    - make start
    script:
    - make test

    View Slide

  41. Use cases
    # DRY - Don’t Repeat Yourself
    # CI/CD Pipeline execution chain
    pipeline: pep8 check_openapi check_k8s unit_test integration_test acceptance_test
    # Deploy application to production
    deploy: cloud_auth build_img push_img migrate_db patch
    @kubectl rollout status deployment $(DEPLOYMENT_NAME)
    @kubectl apply -f $(K8S_DIR)/deployment.yaml

    View Slide

  42. Use cases
    # Getting started
    In order to have the project running locally
    you can just run as follows:
    ```bash
    $> make setup
    ```
    > It will install all project dependencies and
    > and run application using containers
    You can check if everything is OK by running
    the tests:
    ```bash
    $> make test
    ```
    If everything went well, please refer to the
    CONTRIBUTING.md and start contributing to the
    Project!

    View Slide

  43. Use cases
    Kubernetes
    Python language
    NodeJS
    BusyBox
    Docker CLI
    Prometheus
    Traefik
    Terraform

    View Slide

  44. Automating & simplifying

    View Slide

  45. https://github.com/pantuza/makefile
    https://www.gnu.org/software/make/manual/make.html

    View Slide

  46. Desconto de
    50%
    na compra de
    SSL
    Utilize o código
    DEVBR50OFF

    View Slide

  47. https://blog.pantuza.com
    https://github.com/pantuza
    https://twitter.com/gpantuza
    Dúvidas?

    View Slide