Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Shell Script Moderno

Shell Script Moderno

Vídeo dessa palestra:
https://www.youtube.com/watch?v=XBkBnKmu94U

O shell script antigamente era executado em dois ambientes: via linha de comando ou via crontab. Hoje há diversas ferramentas que executam scripts, como Git, Puppet, Jenkins, Travis, GitLab CI e Docker.

Cada ferramenta tem seu próprio ambiente de execução, variáveis e detalhes a se levar em conta. Neste panorama, como fazer scripts robustos, portáveis e funcionais? Quais são os idioms que melhor se encaixam nessa nova realidade?

Palestra apresentada em 2016-07-15 no FISL 2016 (Porto Alegre-RS) http://softwarelivre.org/fisl17

Aurelio Jargas

July 15, 2016
Tweet

More Decks by Aurelio Jargas

Other Decks in Programming

Transcript

  1. Shell Script
    Moderno
    Aurelio Jargas
    http://aurelio.net
    @oreio

    View Slide

  2. Sysadmin anos 90
    Sysadmin anos 90

    View Slide

  3. Sysadmin hoje
    Foto: caminholivre.wordpress.com

    View Slide

  4. Sysadmin dos anos 90
    Servidor físico (empoeirado)
    CD do Linux
    Instala a distro
    Recompila o kernel
    Instala pacotes
    Edita arquivos .conf
    Inicia serviços
    Feito!

    View Slide

  5. Sysadmin dos anos 90
    Servidor físico (empoeirado)
    CD do Linux
    Instala a distro
    Recompila o kernel
    Instala pacotes
    Edita arquivos .conf install.sh
    Inicia serviços
    Feito!
    }

    View Slide

  6. VM
    Cloud
    Docker

    View Slide

  7. Sysadmin / DevOps moderno
    AWS / Azure / Google Cloud
    Puppet / Chef / Ansible
    GitHub / GitLab / Bitbucket
    Jenkins
    Travis CI / GitLab CI
    Docker

    View Slide

  8. Meu precioso servidor,
    criado “na mão”
    com muito ♥

    View Slide

  9. Hoje eu lanço e destruo servidores.
    Com um clique.
    Sem pensar muito.

    View Slide

  10. E o shell?

    View Slide

  11. Ambientes de execução de scripts
    Linha de comando
    $ meu-script.sh foo
    Cron
    0 * * * * root /bin/meu-script.sh foo

    View Slide

  12. Git

    View Slide

  13. Git hooks
    $ ls /path/to/my/repo/.git/hooks/
    applypatch-msg.sample pre-push.sample
    commit-msg.sample pre-rebase.sample
    post-update.sample prepare-commit-msg.sample
    pre-applypatch.sample update.sample
    pre-commit.sample
    $

    View Slide

  14. Git hooks
    $ cat app.bare.git/hooks/post-receive
    #!/bin/bash -exu
    echo "Executando post-receive em producao"
    unset GIT_DIR
    cd /opt/repo/foobar.git
    git checkout producao
    git pull
    $

    View Slide

  15. CI
    Travis, GitLab

    View Slide

  16. Travis CI
    language: bash
    before_install:
    - url_base="https://raw.githubusercontent.com/aureliojargas"
    - curl -sOL "${url_base}/clitest/master/clitest"
    - chmod +x clitest
    - mv clitest testador
    script:
    - cd testador
    - ./run

    View Slide

  17. Travis CI
    language: bash
    before_install:
    - url_base="https://raw.githubusercontent.com/aureliojargas"
    - curl -sOL "${url_base}/clitest/master/clitest"
    - chmod +x clitest
    - mv clitest testador
    script:
    - cd testador
    - ./run

    View Slide

  18. GitLab CI
    code_lint:
    script:
    - date
    - uname -a
    - env
    - ls -la
    - cd wp-content/plugins/foo-setup
    - phpcs .
    - cd -
    - cd wp-content/themes/foo
    - phpcs --ignore=bootstrap,inc,js,css .
    - cd -
    unit_tests:
    script:
    - cd wp-content/plugins/foo-setup
    - WP_TESTS_DIR=/opt/wp-tests/wp-tests-lib phpunit

    View Slide

  19. GitLab CI
    code_lint:
    script:
    - date
    - uname -a
    - env
    - ls -la
    - cd wp-content/plugins/foo-setup
    - phpcs .
    - cd -
    - cd wp-content/themes/foo
    - phpcs --ignore=bootstrap,inc,js,css .
    - cd -
    unit_tests:
    script:
    - cd wp-content/plugins/foo-setup
    - WP_TESTS_DIR=/opt/wp-tests/wp-tests-lib phpunit

    View Slide

  20. Puppet

    View Slide

  21. Puppet
    exec { '/bin/echo root >> /usr/lib/cron/cron.allow':
    path => '/usr/bin:/usr/sbin:/bin',
    unless => 'grep root /usr/lib/cron/cron.allow 2>/dev/null',
    }
    exec { 'tar -xf /Volumes/nfs02/important.tar':
    cwd => '/var/tmp',
    creates => '/var/tmp/myfile',
    path => ['/usr/bin', '/usr/sbin',],
    }

    View Slide

  22. Jenkins

    View Slide

  23. Jenkins

    View Slide

  24. Docker

    View Slide

  25. Docker
    FROM debian:jessie
    ARG DEBIAN_FRONTEND=noninteractive
    RUN apt-get update && apt-get install -y curl
    COPY . /app
    WORKDIR /app
    ENV PATH /app:$PATH
    ENV ZZPATH /app/funcoeszz
    ENV ZZDIR /app/zz
    ENTRYPOINT ["bash", "funcoeszz"]
    CMD ["--help"]

    View Slide

  26. Docker
    FROM debian:jessie
    ARG DEBIAN_FRONTEND=noninteractive
    RUN apt-get update && apt-get install -y curl
    COPY . /app
    WORKDIR /app
    ENV PATH /app:$PATH
    ENV ZZPATH /app/funcoeszz
    ENV ZZDIR /app/zz
    ENTRYPOINT ["bash", "funcoeszz"]
    CMD ["--help"]

    View Slide

  27. Docker
    FROM debian:jessie
    ARG DEBIAN_FRONTEND=noninteractive
    RUN apt-get update && apt-get install -y curl
    COPY . /app
    WORKDIR /app
    ENV PATH /app:$PATH
    ENV ZZPATH /app/funcoeszz
    ENV ZZDIR /app/zz
    ENTRYPOINT ["bash", "funcoeszz"]
    CMD ["--help"]

    View Slide

  28. Docker
    FROM debian:jessie
    ARG DEBIAN_FRONTEND=noninteractive
    RUN apt-get update && apt-get install -y curl
    COPY . /app
    WORKDIR /app
    ENV PATH /app:$PATH
    ENV ZZPATH /app/funcoeszz
    ENV ZZDIR /app/zz
    ENTRYPOINT ["bash", "funcoeszz"]
    CMD ["--help"]
    cp -r . /app
    cd /app

    View Slide

  29. Docker
    FROM debian:jessie
    ARG DEBIAN_FRONTEND=noninteractive
    RUN apt-get update && apt-get install -y curl
    COPY . /app
    WORKDIR /app
    ENV PATH /app:$PATH
    ENV ZZPATH /app/funcoeszz
    ENV ZZDIR /app/zz
    ENTRYPOINT ["bash", "funcoeszz"]
    CMD ["--help"]

    View Slide

  30. Docker
    FROM debian:jessie
    ARG DEBIAN_FRONTEND=noninteractive
    RUN apt-get update && apt-get install -y curl
    COPY . /app
    WORKDIR /app
    ENV PATH /app:$PATH
    ENV ZZPATH /app/funcoeszz
    ENV ZZDIR /app/zz
    ENTRYPOINT ["bash", "funcoeszz"]
    CMD ["--help"]
    export DEBIAN_FRONTEND="noninteractive"
    export PATH="/app:$PATH"
    export ZZPATH="/app/funcoeszz"
    export ZZDIR="/app/zz"

    View Slide

  31. Docker
    FROM debian:jessie
    ARG DEBIAN_FRONTEND=noninteractive
    RUN apt-get update && apt-get install -y curl
    COPY . /app
    WORKDIR /app
    ENV PATH /app:$PATH
    ENV ZZPATH /app/funcoeszz
    ENV ZZDIR /app/zz
    ENTRYPOINT ["bash", "funcoeszz"]
    CMD ["--help"]
    if test $# -gt 0
    then
    bash funcoeszz "$@"
    else
    bash funcoeszz --help
    fi

    View Slide

  32. Boas práticas

    View Slide

  33. Explore o ambiente
    $ date
    $ uname -a
    $ env | sort
    $ whoami
    $ groups
    $ ls -la
    $ ps auxw
    $ git branch
    $ git status
    $ docker info
    $ docker ps

    View Slide

  34. Use funções para organizar o script
    #!/bin/bash
    build() {
    docker build ...
    }
    publish() {
    docker push ...
    docker rmi ...
    }
    run() {
    docker run ...
    }
    build
    publish
    run

    View Slide

  35. Crie uma pasta “scripts” no repositório
    $ ls -1 scripts/
    _lib.sh
    build.sh
    cleanup.sh
    database-reset.sh
    deploy.sh
    merge-branch.sh
    publish.sh
    test.sh
    $

    View Slide

  36. Pratique POSIX
    #!/bin/sh
    Evite arrays
    Evite ${variavel/isso/aquilo}
    Evite ${variavel:1:3}
    Evite [[
    Evite $FUNCNAME
    Evite for(;;)
    Evite $'…'
    Evite comando < <(comando)

    View Slide

  37. Seja --verbose, mande tudo para STDOUT
    echo "Iniciando em $(date)"
    echo "Copiando os arquivos para $destino"
    cp -v * "$destino"
    echo
    echo "Iniciando o backup"
    tar cvzf meuapp-$timestamp.tgz "$destino"
    echo "Finalizando em $(date)"

    View Slide

  38. Nomenclatura de variáveis
    Maiúsculas para variáveis de ambiente:
    $PATH
    $BUILD_ID
    $TRAVIS_OS_NAME
    Minúsculas para variáveis do script:
    base_dir=$(pwd)
    base_url='http://example.com'
    use_colors=1
    Minúsculas para variáveis locais de funções:
    local i=0
    local branch
    local path

    View Slide

  39. Conheça as variáveis do seu ambiente
    Jenkins GitLab CI Travis
    $BUILD_ID
    $BUILD_NUMBER
    $BUILD_URL
    $JOB_NAME
    $GIT_COMMIT
    $CI_BUILD_ID
    $CI_BUILD_NAME
    $CI_BUILD_STAGE
    $CI_SERVER_NAME
    $CI_PROJECT_DIR
    $TRAVIS_BUILD_NUMBER
    $TRAVIS_JOB_NUMBER
    $TRAVIS_COMMIT
    $TRAVIS_OS_NAME
    $TRAVIS_PYTHON_VERSION

    View Slide

  40. Use variáveis de ambiente
    $ ./meu-script.sh meu-app master 123
    $ REPO=meu-app BRANCH=master VERSION=132 ./meu-script.sh

    View Slide

  41. Use o modo “strict” do bash
    #!/bin/bash
    # Modo strict
    set -euo pipefail
    # set -e para abortar o script se houver qualquer erro
    cp /etc/passwd
    # set -u para tornar erro acessar variável não existente
    echo "$foo"
    # set -o pipefail para ajustar o exit code de um pipeline
    grep root /etc/passssswd | sort

    View Slide

  42. Use o modo “strict” do bash (com exceções)
    #!/bin/bash
    # Modo strict
    set -euo pipefail
    # set -e para abortar o script se houver qualquer erro
    cp /etc/passwd || true
    # set -u para tornar erro acessar variável não existente
    echo "${foo:-}"
    # set -o pipefail para ajustar o exit code de um pipeline
    grep root /etc/passssswd | sort

    View Slide

  43. Ligue o debug com $DEBUG
    #!/bin/bash
    test -n "$DEBUG" && set -x
    Para executar:
    $ ./meu-script.sh # sem debug
    $ DEBUG=1 ./meu-script.sh # com debug

    View Slide

  44. Ligue o debug com $DEBUG - strict
    #!/bin/bash
    set -euo pipefail
    test -n "${DEBUG:-}" && set -x

    View Slide

  45. Ferramentas

    View Slide

  46. shellcheck
    http://www.shellcheck.net
    $ shellcheck meu-script.sh
    Line 3:
    for f in $(ls *.txt)
    ^-- SC2045: Iterating over ls output is fragile.
    Line 5:
    grep -i nofx.*mp3 $f
    ^-- SC2062: Quote the grep pattern so the shell won't
    interpret it.
    ^-- SC2086: Double quote to prevent globbing and
    word splitting.

    View Slide

  47. jq - Parser de JSON
    https://stedolan.github.io/jq/
    $ cat foo.json
    { "foo": { "bar": { "baz": 123 } } }
    $ cat foo.json | jq .
    {
    "foo": {
    "bar": {
    "baz": 123
    }
    }
    }
    $ cat foo.json | jq .foo.bar.baz
    123
    $

    View Slide

  48. curl, sed
    $ curl --location -D headers.txt http://www.example.com
    $ sed -i "s/%hostname%/$(hostname)/" /etc/app.conf
    $ sed -i '/debug/ s/false/true/' /etc/app.conf
    $ docker exec meu-app sed -i '/debug/ s/false/true/' /etc/app.conf

    View Slide

  49. Cloud CLI
    AWS
    https://aws.amazon.com/cli/
    aws
    Azure
    https://github.com/Azure/azure-xplat-cli
    azure
    DigitalOcean
    https://github.com/digitalocean/doctl
    doctl
    Google Cloud
    https://cloud.google.com/sdk/
    gcloud, gsutil, bq, kubectl
    IBM Bluemix e Cloud Foundry
    https://clis.ng.bluemix.net
    bluemix, cf
    Rackspace
    https://developer.rackspace.com/docs/rack-cli/
    rack

    View Slide

  50. Perguntas?
    @oreio

    View Slide

  51. $ __

    View Slide

  52. View Slide