Docker - 3 years later

978e79ad01185b39efcfca1482f0f819?s=47 Gijs Molenaar
January 24, 2017

Docker - 3 years later

Here at SKA South Africa we have been dockering for a while now. Although we love docker and think it solves some problems, it also introduces new problems. This talk summarises the problems and our alternatives and workarounds.

978e79ad01185b39efcfca1482f0f819?s=128

Gijs Molenaar

January 24, 2017
Tweet

Transcript

  1. 3 YEARS LATER GIJS MOLENAAR HTTP://PYTHONIC.NL @GIJZELAERR DOCKER

  2. DOCKER @ SKA SA (SCIENCE TEAM) • Experimenting with Docker

    for the last 3 years • Solves various problems • Introduces others • Some problems have been addressed • Others not
  3. WHAT WE TRY TO DO

  4. OUR CURRENT WORKFLOW

  5. OUR APPROACH • Containerise each step • ‘small’ single responsibility

    container • Write metadata per container • Metadata describes parameters, input and output • Combine/chain steps with workflow tool
  6. 2 INHOUSE LIBRARIES • https://github.com/gijzelaerr/ kliko • https://github.com/SpheMakh/ Stimela

  7. None
  8. FUTURE • Focus on off the shelf solutions / standards

    • CommonWL (workflow language) • toil (workflow engine) • toil has Mesos support
  9. CWL ISSUE • doesn’t support random RW file access •

    Copy on write • Sounds weird • Impossible for radio astronomy (TB’s of data) • But it is actually nice • Implicit parallelisation • Reproducibility • Consistency • ‘functional programming’ • Working on getting it in CWL 1.1 • RADIO ASTRONOMY NEEDS NEW DATA FORMAT/ MANAGEMENT
  10. COMMON WORKFLOW LANGUAGE • Abstract interface around command line •

    Defines input, output and parameters • Inline docker definition
  11. None
  12. PROBLEMS WITH DOCKER

  13. ISSUE #6324 DOCKER UNSAFE IN MULTI-TENANT SITUATIONS • Docker daemon

    runs as root • Quite easy to escalate privileges • Bug open for 2.5 years now • https://github.com/docker/ docker/issues/6324 • HPC providers don’t allow docker because of this
  14. MANAGING PERMISSIONS OSX UID Mapping Naive

  15. WORKAROUNDS • The abstraction just doesn’t work very well •

    pass UID env variable to container, do post chown • create a world writeable work folder and do UID mapping, override HOME variable
  16. THE ALTERNATIVE • Keep asking yourself: do you *REALLY* need

    docker? • Why no packaging? Build scripts? Fabric? Ansible? The internet was created without docker. • Alternative solutions! Singularity, rkt, LXC • Just have a look at it
  17. None
  18. None
  19. SINGULARITY • Less magic than docker • image file based

    • gets the HPC job done • Most likely candidate to get deployed on clusters • Also needs workarounds for GPU acceleration :/
  20. GPU ACCELERATION (CUDA) • problem: libraries need to match kernel

    version • workaround: nvidia-docker • workaround: custom container per Nvidia module • Replacing one problem with an other one! Aks yourself why do you need docker?
  21. WHY MY CONTAINER HUGE WHY DOES IT TAKE SO LONG

    TO BUILD - HOW DO I MIX CONTAINERS
  22. DO PACKAGE MANAGEMENT • Why are you reinventing the wheel?

    • Docker is not a package manager • Make packages! • Making installation of software easier since 1993 • deb / RPM, whatever, make packages
  23. ADVANTAGES • Get working once, use everywhere • Use in

    any container solution, or VM. Or host! • Combine packages in new containers • Manage dependencies • There are many tools out there to help you packaging • You can automate packaging
  24. None
  25. KERNSUITE.INFO

  26. None
  27. DOCKER 1.13

  28. PACKAGING HOW TO PART 2

  29. GNU/DEBIAN • Ubuntu is based on GNU/Debian • We use

    Ubuntu 14.04 / 16.04 • Build / Host packaging on ‘the cloud’ • Launchpad PPA
  30. WSCLEAN • https://sourceforge.net/projects/wsclean • https://github.com/kernsuite-debian/wsclean • http://packages.ubuntu.com/zesty/wsclean

  31. ANATOMY OF WSCLEAN

  32. ANATOMY OF A DEBIAN PACKAGE

  33. ANATOMY OF A DEBIAN SOURCE PACKAGE

  34. ANATOMY OF THE DEBIAN FOLDER

  35. None
  36. None
  37. LAUNCHPAD PPA

  38. UPLOADING TO PPA USING DPUT

  39. BUILD THE PACKAGE IN ‘THE CLOUD’

  40. GIT-BUILDPACKAGE • Use git for managing the packaging • Mirror

    original released software in git • use branches/tags for version indication • Augment source tree with Debian files • use gbp CLI to manage Debian source tree • https://github.com/kernsuite-debian/wsclean
  41. MORE LINKS • Ubuntu packaging guide: http://packaging.ubuntu.com/ (getting set up)

    • packaging on steroids with git-buildpackage: http:// honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.html • how we do it: https://github.com/kernsuite/packaging/wiki/How- to-do-KERN-packaging • template: https://github.com/kernsuite/template • the official Debian documentation: https://www.debian.org/doc/ manuals/maint-guide/
  42. ROUNDING UP • Debian packaging is not so hard if

    the packaged software is well written • Many implicit helper functions • Makes it a bit harder when software misbehaves • Do the packaging on a clean system/environment (docker!)
  43. CONCLUSIONS • Docker is cool, and changed a lot of

    things • Docker is not suited for all cases • Some problems can/will be solved but not all • singularity probably better for shared infrastructure • Packaging software is almost always a good thing
  44. QUESTIONS? Gijs Molenaar - http://pythonic.nl - @gijzelaerr