docker-record
 A Semi-Automated Approach from Container Setup to Dockerfile Jürgen Cito Photo Credits: Nan Palmero, https://flic.kr/p/nPLSpe 

Docker Hackathon in New York City 
 WINNER: docker-record 
 @citostyle, @allonhadaya Tool to address shortcoming in setup and maintenance process Open Source on GitHub: https://github.com/citostyle/docker-record 

Dockerfile Definition of infrastructure and dependencies of a container through instructions # Build redis from source # Make sure you have the redis source code checked out in # the same directory as this Dockerfile FROM ubuntu:12.04 MAINTAINER dockerfiles http://dockerfiles.github.io RUN echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/ sources.list RUN apt-get update RUN apt-get upgrade -y RUN apt-get install -y gcc make g++ build-essential libc6-dev tcl wget RUN wget http://download.redis.io/redis-stable.tar.gz -O - | tar -xvz # RUN tar -zvzf /redis/redis-stable.tar.gz RUN (cd /redis-stable && make) RUN (cd /redis-stable && make test) RUN mkdir -p /redis-data VOLUME ["/redis-data"] EXPOSE 6379 ENTRYPOINT ["/redis-stable/src/redis-server"] CMD ["--dir", "/redis-data"] Dependencies Base Image Install Open Port Start Server 

The Ideal Container World App Idea A Dockerfile magically appears # Build redis from source # Make sure you have the redis source code checked out in # the same directory as this Dockerfile FROM ubuntu:12.04 MAINTAINER dockerfiles http://dockerfiles.github.io RUN echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/ sources.list RUN apt-get update RUN apt-get upgrade -y RUN apt-get install -y gcc make g++ build-essential libc6-dev tcl wget RUN wget http://download.redis.io/redis-stable.tar.gz -O - | tar -xvz # RUN tar -zvzf /redis/redis-stable.tar.gz RUN (cd /redis-stable && make) RUN (cd /redis-stable && make test) RUN mkdir -p /redis-data VOLUME ["/redis-data"] EXPOSE 6379 ENTRYPOINT ["/redis-stable/src/redis-server"] CMD ["--dir", "/redis-data"] Build Image Run Containers ??? Profit 

The Real Container World # Adding some config to nginx FROM nginx:1.9.5 We need certificates!! # Install wget and install/updates certificates RUN apt-get update \ && apt-get install -y -q --no-install-recommends \ ca-certificates \ wget \ && apt-get clean \ Build new image Run new container Run Tests, Trial&Error, … 

The Real Container World # Adding some config to nginx FROM nginx:1.9.5 Test fail Server names are too long # Install wget and install/updates certificates RUN apt-get update \ && apt-get install -y -q --no-install-recommends \ ca-certificates \ wget \ && apt-get clean \ Build new image Run new container Run Tests, Trial&Error, … # Configure Nginx and apply fix for very long server names RUN echo "daemon off;" >> /etc/nginx/nginx.conf \ && sed -i 's/^http {/&\n server_names_hash_bucket_size 128;/g' /etc/nginx/ nginx.conf 

The Real Container World # Adding some config to nginx FROM nginx:1.9.5 Test fail Volume for certificates # Install wget and install/updates certificates RUN apt-get update \ && apt-get install -y -q --no-install-recommends \ ca-certificates \ wget \ && apt-get clean \ Build new image Run new container Run Tests, Trial&Error, … # Configure Nginx and apply fix for very long server names RUN echo "daemon off;" >> /etc/nginx/nginx.conf \ && sed -i 's/^http {/&\n server_names_hash_bucket_size 128;/g' /etc/nginx/ nginx.conf ... VOLUME ["/etc/nginx/certs"] ... 

Container Setup & Maintenance Initial Setup Configuration Tuning in Container Run Tests, Trial&Error Helplessly identify what has actually been done within the container and what files have been touched and translate it to Dockerfile instructions. Test the new image, pray it worked out 

docker-record To The Rescue! Photo Credits: José María Pérez Nuñez, https://flic.kr/p/ksiHGZ 

Usage docker-record container Container should be running first Starts an interactive (bash) session and ‘records’ what you are doing 

Usage docker-record container --replay Generates Dockerfile by ‘replaying’ your previously recorded sessions 

Behind the Scenes Photo Credits: Jonathan Kos-Read, https://flic.kr/p/cepP61 

Instrumentation For every started session we gather the execution context > User Id > Bash Command > Working Directory > Environment Variables 

Instrumentation docker exec -ti container bash -c "bash --init-file <(echo ‘SESSION_ROOT=/tmp/record; mkdir -p $SESSION_ROOT; function __instrument() { echo $EUID >> $SESSION_ROOT/euid && echo $BASH_COMMAND >> $SESSION_ROOT/cmd && echo $PWD >> $SESSION_ROOT/pwd && echo $(export | tr "\\n" ";") >> $SESSION_ROOT/env || exit; }; shopt -s extdebug; trap __instrument DEBUG;'')" 

Instrumentation docker exec -ti container bash -c "bash --init-file <(echo ‘SESSION_ROOT=/tmp/record; mkdir -p $SESSION_ROOT; function __instrument() { echo $EUID >> $SESSION_ROOT/euid && echo $BASH_COMMAND >> $SESSION_ROOT/cmd && echo $PWD >> $SESSION_ROOT/pwd && echo $(export | tr "\\n" ";") >> $SESSION_ROOT/env || exit; }; shopt -s extdebug; trap __instrument DEBUG;'')" Start bash within bash an pass init ‘file’ 

Instrumentation docker exec -ti container bash -c "bash --init-file <(echo ‘SESSION_ROOT=/tmp/record; mkdir -p $SESSION_ROOT; function __instrument() { echo $EUID >> $SESSION_ROOT/euid && echo $BASH_COMMAND >> $SESSION_ROOT/cmd && echo $PWD >> $SESSION_ROOT/pwd && echo $(export | tr "\\n" ";") >> $SESSION_ROOT/env || exit; }; shopt -s extdebug; trap __instrument DEBUG;'')" Define function within ‘init file’ 

Record - Instrumentation docker exec -ti container bash -c "bash --init-file <(echo ‘SESSION_ROOT=/tmp/record; mkdir -p $SESSION_ROOT; function __instrument() { echo $EUID >> $SESSION_ROOT/euid && echo $BASH_COMMAND >> $SESSION_ROOT/cmd && echo $PWD >> $SESSION_ROOT/pwd && echo $(export | tr "\\n" ";") >> $SESSION_ROOT/env || exit; }; shopt -s extdebug; trap __instrument DEBUG;'')" Record context in temporary files 

Record - Instrumentation docker exec -ti container bash -c "bash --init-file <(echo ‘SESSION_ROOT=/tmp/record; mkdir -p $SESSION_ROOT; function __instrument() { echo $EUID >> $SESSION_ROOT/euid && echo $BASH_COMMAND >> $SESSION_ROOT/cmd && echo $PWD >> $SESSION_ROOT/pwd && echo $(export | tr "\\n" ";") >> $SESSION_ROOT/env || exit; }; shopt -s extdebug; trap __instrument DEBUG;'')" Debug mode ‘trap’ catches signal and executes our function 

Replay - Extraction & Filter > Not all bash commands lead to Dockerfile statement > Blacklisting, Whitelisting > Heuristics for mapping execution context 

Replay - Mapping Heuristics (1/3) CMD instruction - there is only one in a Dockerfile > Defines an exectuable for the start of a container > Whitelisting paths that are likely to contain an executable 

Replay - Mapping Heuristics (2/3) Match Editors that have been used - indicates a file that has been changed > Whitelist for possible editors > Check whether opened files have been actually edited in container > Copy file from container and add an ADD instruction 

Replay - Mapping Heuristics (3/3) Everything else that hasn’t been filtered gets executed > RUN instruction for every remaining extracted command 

docker-record in Action Walking through a tutorial by DigitalOcean - Screencast: vimeo.com/139944015 docker run -ti --name=demo -p 80:80 ubuntu /bin/bash 1. Run container (image: Ubuntu, container name: demo) 

docker-record in Action Walking through a tutorial by DigitalOcean - Screencast: vimeo.com/139944015 docker-record demo
 root@2208baba1afc:/# 2. Start interactive shell with docker-record (Every command you issue now will be instrumented) 

docker-record in Action Walking through a tutorial by DigitalOcean - Screencast: vimeo.com/139944015 https://www.digitalocean.com/community/tutorials/docker-explained-how-to-containerize-and-use-nginx-as-a-proxy 3. Follow the tutorial to install and configure nginx 

docker-record in Action Walking through a tutorial by DigitalOcean - Screencast: vimeo.com/139944015 docker-record demo --replay 4. Exit container and generate Dockerfile RUN apt-get update RUN apt-get install -y nano wget dialog net-tools RUN apt-get upgrade -y nginx RUN rm -v /etc/nginx/nginx.conf ADD ./build/_etc_nginx_nginx.conf /etc/nginx/nginx.conf CMD service nginx start 

Future Work Heuristics Domain Knowledge
 > Better filter to suggest build context
 > Suggesting VOLUMES Dependency Analysis > Reordering/grouping of instructions 

Future Work
 Refactoring, Refactoring, Refactoring… # make it work now, refactor later 

Supported by the European Community’s Seventh Framework Programme (FP7/2007-2013) under grant agreement no. 610802 (CloudWave). www.cloudwave-fp7.eu 

docker-record
 A Semi-Automated Approach from Container Setup to Dockerfile Jürgen Cito Photo Credits: Nan Palmero, https://flic.kr/p/nPLSpe @citostyle Slides: speakerdeck.com/citostyle https://github.com/citostyle/docker-record