Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Thinking inside the box: Dockerizing Perl

Thinking inside the box: Dockerizing Perl

Docker provides a lightweight environment for running Perl, but the documentation for starts with a full copy of Ubuntu! Instead of treating docker containers as VM's we can build a minimal container that works as a base for running applications with Perl.

This talk describes the basics of building a new container with a few alternatives for using an O/S image, busybox, libraries, or nothing to run Perl.

Steven Lembark
PRO

July 09, 2022
Tweet

More Decks by Steven Lembark

Other Decks in Technology

Transcript

  1. Thinking inside the box: Dockerizing Perl
    Steven Lembark
    Workhorse Computing
    [email protected]

    View Slide

  2. What's on your box?
    Pre-compiled perl distros are terrible.
    Bulky, slow, with annoying dependencies.
    Not what you need...

    View Slide

  3. Example: Red Hat Enterprize 7.0
    5.16 == “end of life”
    -Doptimize=-O2 -g -pipe -fstack-
    protector-strong -mtune=generic -
    Dversion=5.16.3 -Dmyhostname=localhost
    [email protected] -
    Dusethreads -Duseithreads -
    Ubincompat5005

    View Slide

  4. Example: Red Hat Enterprize 7.0
    5.16 == “end of life”
    Not exactly built for speed.
    -Doptimize=-O2 -g -pipe -fstack-
    protector-strong -mtune=generic -
    Dversion=5.16.3 -Dmyhostname=localhost
    [email protected] -
    Dusethreads -Duseithreads -
    Ubincompat5005

    View Slide

  5. Example: Red Hat Enterprize 7.0
    5.16 == “end of life”
    Not exactly built for speed.
    Thread overhead, even if you don't use them.
    -Doptimize=-O2 -g -pipe -fstack-
    protector-strong -mtune=generic -
    Dversion=5.16.3 -Dmyhostname=localhost
    [email protected] -
    Dusethreads -Duseithreads -
    Ubincompat5005

    View Slide

  6. Fresh, not frozen
    perl for your architecture.
    optimized.
    only dependencies you use (need).
    Q: How?

    View Slide

  7. Try it, you'll like it...
    Bad approach: virtual machines...

    View Slide

  8. Try it, you'll like it...
    Bad approach: virtual machines...
    and version-dirs...
    and recompiling for each distro, architecture...
    and symlink hell(2)...
    and …

    View Slide

  9. Try it, you'll like it...
    False lazyness!

    View Slide

  10. Tasty alternative: lxc
    Essentially an LPAR – we've come full circle.
    Use a process to isolate & localize code.
    Share the kernel.
    Light weight, fast startup, easy to ship.

    View Slide

  11. Fly in the soup
    Ever try to use lxc?
    Let alone finish the manpages?
    Kills my appetite.
    RTFM
    lxc-attach.1
    lxc-autostart.1
    lxc-cgroup.1
    lxc-checkconfig.1
    lxc-checkpoint.1
    lxc-clone.1
    lxc-config.1
    lxc-console.1
    lxc-create.1
    lxc-destroy.1
    lxc-device.1
    lxc-execute.1
    lxc-freeze.1
    lxc-info.1
    lxc-ls.1
    lxc-monitor.1
    lxc-snapshot.1
    lxc-start-ephemeral.1
    lxc-start.1
    lxc-stop.1
    lxc-top.1
    lxc-unfreeze.1
    lxc-unshare.1
    lxc-user-nic.1
    lxc-usernet.5
    lxc-usernsexec.1
    lxc-wait.1
    lxc.7
    lxc.conf.5
    lxc.container.conf.5
    lxc.system.conf.5

    View Slide

  12. Docker: MRE for linux
    80/20 of lxc:
    Layered filesystem + Repository + Command line.
    More nutritious than exciting.
    Still evolving.

    View Slide

  13. Catch: Docker's Docs aren't lean!
    Docker doc's start with ubuntu image.
    1.6GiB – Heart attack on a plate!
    Includes X11 libs, lvm, mdadm, grub, parted... perl.

    View Slide

  14. A: There is more than one way to do it.
    Dockerfile builds perl on a light[er] weight O/S.
    Single package for perl + shared libs.
    Shell tools available for qx{...}.
    Still pretty heavy-weight.

    View Slide

  15. A: There is more than one way to do it.
    Copy perl on top of busybox.
    Much leaner cuisine.
    Decent collection of shell tools.
    Shared lib's as layer or via -v.

    View Slide

  16. A: There is more than one way to do it.
    Just perl
    No empty calories.
    Dodge library issues with -v or static compile.
    Lacks tools to inspect the build.

    View Slide

  17. Start by getting docker.
    They provide shell code for the basics:
    $curl -sL https://get.docker.io/ | sh;
    $wget -qO- https://get.docker.io/ | sh;
    will do the deed on Fedora, Ubuntu/Debian, or Gentoo.
    Also need to sanity check the kernel.

    View Slide

  18. Be yourself
    Don't run as su.
    Add your users to “docker” in /etc/group.
    After that check that docker is running:
    $ docker ps;
    Get access to the repository:
    $ docker login;

    View Slide

  19. Just a taste...
    Minimal base container: busybox.
    Good for experimenting:
    $ docker pull busybox
    Pulling repository busybox
    fd5373b3d938: Download complete
    ...
    f06b02872d52: Download complete

    View Slide

  20. Getting inside the box
    # /bin/sh is entrypoint
    $ docker run -t -i busybox;
    # <-- su in box, login id out.
    # ping 8.8.8.8; <-- network available.
    ...
    # exit; <-- exit docker process
    $ <-- original EUID

    View Slide

  21. Gentoo is easy to dockerize
    Common solution is a “stage-3” system.
    Shell + libs + build tools.
    Not much else.
    About half the size of Ubuntu.

    View Slide

  22. Finding a distribution
    Start at the docker registry
    https://registry.hub.docker.com/
    Looking for stage-3 builds:
    https://registry.hub.docker.com/search?q=gentoo+stage3
    I find:
    jgkim/gentoo-stage3 741.2 MB
    Reasonable start.

    View Slide

  23. Grabbing an O/S
    Get the image:
    $ docker pull jgkim/gentoo-stage3;
    Run a container:
    $ docker run –rm -i -t jgkim/gentoo-stage3;
    # gcc --version;
    gcc 4.8.4 good supports “--arch=native”

    View Slide

  24. Building a perl container
    Github has templates:
    http://github.com/Perl/docker-perl
    Dockerfiles like “5.020.000-64bit/Dockerfile”.
    git acquires “5.02.0-64bit-optimized” directory.

    View Slide

  25. FROM buildpack-deps # parent container
    RUN apt-get update && apt-get install -y curl procps # commands to pull perl
    RUN mkdir /usr/src/perl
    WORKDIR /usr/src/perl # build dir within the container
    RUN curl -SL http://www.cpan.org/src/5.0/perl-5.20.0.tar.gz | tar -xz --strip-components=1
    RUN ./Configure -Duse64bitall -des \
    && make -j$(nproc) && TEST_JOBS=$(nproc) make test_harness \
    && make install && make veryclean
    WORKDIR /root
    CMD ["perl5.20.0","-de0"] # /bin/sh perl5.20.0 -de0
    Distro's Dockerfile

    View Slide

  26. Check the local arguments
    $ perl -V;
    ...
    config_args='-de -Dprefix=/opt/perl/5.20 -
    Doptimize=-O3 -march=native -pipe'
    perl -MConfig -E 'say $Config{ config_args }'

    View Slide

  27. New Dockerfile
    FROM jgkim/gentoo-stage3
    MAINTAINER Steven Lembark
    WORKDIR /var/tmp/
    RUN wget http://www.cpan.org/src/5.0/perl-5.20.2.tar.gz;
    RUN gzip -dc < perl-5.20.2.tar.gz | tar xf -;
    RUN cd perl-5.20.2 && \
    ./Configure -de -Dprefix=/opt/perl \
    -Dman1dir=none -Dman3dir=none \
    -Doptimize='-O3 -march=native -pipe' ;
    RUN make -C perl-5.20.2 all test install distclean;
    RUN /opt/perl/bin/h2ph -r -l /usr/include;
    CMD [ "/opt/perl/bin/perl", "-d", "-E", "42" ]

    View Slide

  28. Building Perl
    The build takes input and optional repository tag.
    Input is a directory not “Dockerfile”
    $ cd /scratch/docker/gentoo+perl;
    $ docker build –tag='lembark/perl-gentoo' . ;
    Each step in the Dockerfile is an intermediate image.

    View Slide

  29. Checking local containers
    Intermediate images labled with “”.
    These were from prior tests.
    This one didn't finish:
    $ docker images
    REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
    5906d8edbb59 10 minutes ago 726.1 MB
    10f5517fcada 23 minutes ago 726.1 MB
    7d328e761704 26 minutes ago 725.5 MB
    88f1e41aaed5 27 minutes ago 725.5 MB
    28052ace7e04 28 minutes ago 725.5 MB
    6f46836220d9 31 minutes ago 725.5 MB

    View Slide

  30. $ docker build --rm=false –tag=lembark/perl:5.20 .;
    Step 0 : FROM rndevfx/gentoo-stage3-amd64-nomultilib
    ---> e9d0ce66148c
    Step 1 : MAINTAINER Steven Lembark
    ---> Using cache <-- recycled image
    ---> 41d5480e49e7
    Step 2 : WORKDIR /var/tmp
    ---> Using cache
    ---> 010d3d70ced1
    ...
    Step 7 : RUN make all test install; <-- first execution is here.
    ---> Running in 949c9ddfc2ef
    Step 8 : RUN /opt/perl/bin/h2ph -r -a -l;
    ---> Running in b084569e8fc3
    -r and -a options are mutually exclusive
    INFO[1342] The command [/bin/sh -c /opt/perl/bin/h2ph -r -a -l;] returned a non-zero code
    oops...

    View Slide

  31. Easy fix
    RUN /opt/perl/bin/h2ph -r -l /usr/inlcude;

    View Slide

  32. True Lazyness
    Re-running “build” re-cycles images:
    ...
    Step 7 : RUN make all test install;
    ---> Using cache
    ---> cb4c5cd0607e
    Step 8 : RUN /opt/perl/bin/h2ph -r -l /usr/include;
    ---> Running in 92efee8c31e2
    require '_h2ph_pre.ph';
    ...

    View Slide

  33. And impatience
    $ time docker build --rm=false --tag=lembark/perl-gentoo .
    Sending build context to Docker daemon 4.096 kB
    Sending build context to Docker daemon
    Step 0 : FROM rndevfx/gentoo-stage3-amd64-nomultilib
    ...
    Step 10 : CMD [ “/opt/perl/bin/perl” ]
    ---> Using cache
    ---> f7b83ecbe276
    Successfully built f7b83ecbe276
    real 0m0.241s
    user 0m0.019s
    sys 0m0.012

    View Slide

  34. Docker Images:
    REPOSITORY VIRTUAL SIZE
    lemark/gentoo-perl 884.8 MB
    jgkim/gentoo-stage3 741.2 MB
    localhost:5000/lembark/busybox 1.9 MB
    Adding “RUN rm -rf perl-5.20.2”:
    lemark/gentoo-perl 884.8 MB
    Cannot reclaim build space!
    Successful build: tagged image

    View Slide

  35. Welcome to perl
    “CMD” gets run with an interactive container:
    $ docker run –rm -i -t lembark/gentoo-perl;
    Loading DB routines from perl5db.pl version 1.44
    Editor support available.
    ...
    main::(-e:1): 0
    DB<1> x $^V
    v5.20.2

    View Slide

  36. Choose your toppings
    Default for gentoo run /bin/bash .
    Save typing with:
    ENTRYPOINT [ “/opt/perl/bin/perl” ]
    CMD [ "-d", "-E", "42" ]
    And then:
    docker run –rm -i -t lembark/docker-perl;
    Use –entrypoint='/bin/bash' if perl fails.

    View Slide

  37. The next course
    Stacked images inherit the ENTRYPOINT:
    FROM lembark/gentoo-perl
    CMD [ "/path/to/your/perl_code" ]

    View Slide

  38. Test containers stack
    Derive tests from the package.
    Add ./t to another image.
    ENTRYPOINT [ "/opt/perl/bin/prove" ]
    Result: no tests in product image:
    docker run foo/bar; run application.
    docker run foo/bar-test; run base tests.

    View Slide

  39. Lighter, reasonably tasty...
    But still way, way too high-calorie.
    You already have a local O/S.
    Q: Why add one to the container?
    A: Because we are all used to virtual machines.
    Time for really lean cuisine.

    View Slide

  40. Copy perl on top of busybox
    Build & install into /opt/perl.
    Add /opt/perl/Dockerfile:
    FROM lembark/busybox_x86
    COPY [ “.”, “/opt/perl” ]
    $ docker build /opt/perl lembark/busybox-perl;

    View Slide

  41. Nice combination
    Useful shell tools for "qx".
    Examine broken perl with /bin/sh entrypoint.

    View Slide

  42. Nothing artificial, nothing added
    Build a naked /opt/perl.
    Leave out “busybox”:
    FROM lembark/empty;
    COPY [ “.”, “/opt/perl” ]
    Minimal base for Plack or coros.

    View Slide

  43. Shared libs
    Catch: perl usually needs them.
    In my case, from /lib64.
    One way: O/S base image.
    Q: How else?

    View Slide

  44. Fix: Share libs
    Run containers with read-only mount of /lib64:
    docker run -v /lib64:/lib64:r …
    Light-weight.
    Fast: No images to ship.
    Homogenous lib's.

    View Slide

  45. Fix: Add libs
    “ldd” lists shared libs.
    COPY them into the image with perl.
    Build perl from a lib64 image.
    linux-vdso.so.1 (0x00007ffdfbbcb000)
    libperl.so => /opt/perl/5.20/lib/5.20.2/x86_64-linux/CORE/libperl.so
    (0x00007f40f8868000)
    libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f40f8650000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00007f40f844c000)
    libm.so.6 => /lib64/libm.so.6 (0x00007f40f8153000)
    libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f40f7f1c000)
    libutil.so.1 => /lib64/libutil.so.1 (0x00007f40f7d19000)
    libc.so.6 => /lib64/libc.so.6 (0x00007f40f7981000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f40f8c2b000)

    View Slide

  46. Fix: static perl
    Add “--static”.
    No hetergenious server issues.
    perl image is larger.
    Container has same virtual size.
    Best for tests: no issues with underlying images.

    View Slide

  47. Result: portable, optimized, minimal perl.
    Static perl with busybox: < 250 MiB.
    Including Dancer, mod_perl, and friends.
    Whole lot less than Ubuntu.
    Not a virtual machine.
    Plack web server FROM this perl.
    Viola!, you're up.

    View Slide

  48. References
    http://docs.docker.com/
    .../builder Dockerfile directives
    .../userguide What you think it is.

    View Slide

  49. Summary
    Docker makes lxc approachable.
    perl on busybox is tasty, low-calorie.
    Add /lib64 for a complete meal.

    View Slide