Share
Embed
Singularity Containers & The Rewrite to Go
Presentation of what Singularity Containers are and what we learned from our move to Go.
Andre Marcelo-Tanner
December 18, 2018
Transcript
24 slides
-
Singularity A brief history • Invented by Greg Kurtzer at
LBNL to address problems specific to HPC • Development began in October 2015 • First release April 2016 • User feedback, major changes and help from the community leads to 2.0 in June 2016 • Greg founds Sylabs.io January 1, 2018; already at 28 employees, product and revenue! • Sylabs developers completely overhaul code base, create a suite of cloud services, and release 3.0 October 8, 2018! -
Singularity is Designed Specifically for HPC • Compatible with traditional
HPC architecture and workflows • Supports all known resource managers • Optimized for compute jobs and performance • Supports GPUs, IB, OmniPath, MPI, parallel file systems, production kernels, etc... • Users are themselves within the container • Allows users to “BYOE” • Blocks privilege escalation within the container -
Simplicity Singularity is designed to be simple. From its build
syntax, to its image format, the runtime, and compatibility with OCI and Docker, Singularity facilitates easy and intuitive onboarding. SIF can squashe, encapsulate and sign Docker and OCI image formats (encryption coming soon) Docker, OCI, etc... Singularity -
Reproducibility and Mobility Singularity uses binary and immutable images, encapsulating
the entire runtime stack so it is always 100% reproducible. Host 1 Host 2 SCP, FTP, HTTP, Sylabs Cloud -
Container Image Integrity Singularity container runtimes are immutable, cryptographically signed,
and verifiable, ensuring absolute trust of the container environment. Note: Singularity uses no tarballs and thus no unsigned intermediate data -
Designed for HPC and up! From your laptop, to HPC
compute, to the cloud and edge, Singularity facilitates extreme mobility of compute. Singularity provides an onramp across all resources, enterprises, and clouds. -
Widely Adopted for Compute Needs Singularity is the container runtime
of choice for compute based workloads like simulation and AI As of Singularity 2.x… • Millions container runs per day • Estimated user base of more than 25k • Installed on over 3 million sockets HPCWire Readers and Editors Choice Awards: • 2016: Top products to watch • 2017: Top products to watch • 2017: Best HPC Programming Tool/Tech The Uptake of Singularity has been even faster than CentOS! -
$ cat recipe.def Bootstrap: yum OSVersion: 7 MirrorURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/os/x86_64/ Include:
yum %post yum -y install vim-minimal %runscript exec /usr/bin/vi “$@” $ singularity build centos-vim.sif recipe.def … Building a Container with Singularity -
# You can use the container as follows... $ singularity
shell centos-vim.sif $ singularity exec centos-vim.sif vim testfile.txt $ singularity run centos-vim.sif testfile.txt # Singularity containers are also executable, so you can ‘run’ them directly $ ./centos-vim.sif testfile.txt # You can move the container onto any other Linux system with Singularity installed, and use the container directly $ scp centos-vim.sif [email protected]: $ ssh [email protected] $ ./centos-vim.sif newtestfile.txt Example Usage of Singularity -
$ singularity sign centos-vim.sif Signing image: centos-vim.sif No OpenPGP signing
keys found, autogenerate? [Y/n] y Enter your name (e.g., John Doe) : Greg Enter your email address (e.g., [email protected]) : [email protected] Enter optional comment (e.g., development keys) : demokeys Generating Entity and OpenPGP Key Pair... Done Enter encryption passphrase : … Uploaded key successfully! Enter key passphrase: Signature created and applied to centos-vim.sif $ Cryptographically signed containers -
$ singularity push centos-vim.sif library://gmk/demo/centos-vim:latest INFO: Now uploading centos-vim.sif to
the library 108.16 MiB / 108.16 MiB [=============================] 100.00% 13.75 MiB/s 7s INFO: Setting tag latest $ Using the Sylabs Container Library -
$ singularity pull library://gmk/demo/centos-vim:latest 108.16 MiB / 108.16 MiB [=============================]
100.00% 35.00 MiB/s 3s $ singularity verify centos-vim_latest.sif Verifying image: centos-vim_latest.sif INFO: key missing, searching key server for KeyID: 58D8405A30E12DE6... INFO: key retreived successfully! Store new public key F56D95BD3AFAC6FA3423911A58D8405A30E12DE6? [Y/n] y Data integrity checked, authentic and signed by: Greg (demokeys) <[email protected]>, KeyID 58D8405A30E12DE6 $ Pulling and Validating a Container -
# Building a Singularity container (SIF) from DockerHub $ singularity
build python.sif docker://python:latest … # Running a shell directly from DockerHub $ singularity shell docker://ubuntu:latest Singularity ubuntu_latest.sif:~/demo> cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=18.04 DISTRIB_CODENAME=bionic DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS" Singularity ubuntu_latest.sif:~/demo> exit $ singularity exec docker://centos:latest cat /etc/redhat-release CentOS Linux release 7.5.1804 (Core) Working with Docker -
$ singularity exec --nv docker://tensorflow/tensorflow:latest-gpu python Python 2.7.12 (default, Dec
4 2017, 14:50:18) [GCC 5.4.0 20160609] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tensorflow as tf >>> x1 = tf.constant([1,2,3,4]) >>> x2 = tf.constant([5,6,7,8]) >>> result = tf.multiply(x1, x2) >>> print(result) Tensor("Mul:0", shape=(4,), dtype=int32) >>> exit() $ Accessing the Host’s GPU With Tensorflow -
Rewrite to Golang Feb 2018, Released 3.0 in Golang on
October 2018 https://www.sylabs.io/2018/02/singularity-golang/ -
Wins & Gains from converting to Go • Integration with
other container projects. Easy to use packages from Kubernetes, Docker/Moby, OCI etc. No need to rebuild in Python or C. • Concurrency model of Go. • Go is easy to learn for developers coming from other languages. • We used CNI rather than creating our own networking stack. -
Challenges using/converting to Go • C-Go Interface, used an RPC
server to communicate between C and Go binaries. • Forking in Go. Certain syscalls should not be called in a multi threaded application. • Not following Go Standards (packaging, internal) • Vendoring… dep, go mod, vendor/ • Packaging. Custom packaging tool makeit. -
Comparing Go to previous Languages (Python etc) • Large ecosystem
of packages for backend and system projects, cloud APIs. • go test - Easy standards and framework for setting up tests. • Easy to be productive in Go. Well designed language. • Go is opinionated (Good and Bad). Works well but you have to follow the standards. Easy to read other codebases. -
In hindsight • Focus on strong core and plan to
make project modular. • Refactoring… • Better package planning. • Keep integration/acceptance tests in old format. Use go test for unit tests. Maybe use bats for integration tests.