Containerize your Python apps like it's 2024

There are a lot of resources on containerizing Python applications with Docker, but most are basic and outdated. Following them results in slow builds and potentially insecure applications. Let’s see how we can build better containers using recent Docker features!

This talk will show how to speed up your builds and make your images smaller and more secure. We’ll use features such as multi-stage builds or cache mounts to build containers with Python apps. We will also discuss how to improve the security of your container.

Tips from the talk are valid for applications of all sizes and kinds: from hobby projects to enterprises, from CLI tools to web applications and APIs. You will be able to apply them immediately after the talk.

Basic knowledge of Docker and its key concepts (images, layers, Dockerfile commands) is required. You’ll learn something new even if you have used Docker for some time.

Transcript

1. infrastruktura webová řešení marketing Containerize your Python apps like it's 2024 Jan Smitka
2. Distribute the app + its environment 2
3. Production Image 3 
4. Fast Builds 4  
5. Small 5  
6. Secure 6 
7.   Every project is di ff erent 7   
8. Base image 8 
9. Clean system Install Python yourself 9 
11. python:* - alpine Very small 11 
12. musl libc: slower memory allocator 12 Performance comparison musl glibc Tiny allocation & free 0.005 0.002 Big allocation & free 0.027 0.016 https://www.etalabs.net/compare_libcs.html
13. Availability of binary wheels 13
14. python:* - bookworm Lot of packages preinstalled 14 
15.  python:* - slim - bookworm Minimized 15 
16. Version pinning Good python:3.12.3-slim - bookworm python:3.12-slim - bookworm Bad python:3.12 16  
17. Image Layers 17
18. Layer Cache and Rebuild 18
19. Layer Cache and Rebuild 19
20. Layer Order 20
21. Installing build dependencies FROM python:3.12-slim - bookworm RUN apt - get update & & apt - get install - y \ default - libmysqlclient - dev \ build - essential \ pkg - conf i g \ & & rm - rf /var/lib/apt/lists / * COPY requirements.txt ./ RUN pip install - r requirements.txt # . . . 21 0 upgraded, 58 newly installed, … Need to get 69.7 MB of archives. After this operation, 313 MB of additional disk space will be used.
22. Multi-stage builds 22 
23. Multi-stage builds FROM python:3.12-slim - bookworm AS builder RUN python - m venv /opt/venv # TODO : Install dependencies FROM python:3.12-slim - bookworm AS production COPY - - from=builder /opt/venv /opt/venv # . . . 23
24. docker build - - target=production … 24 Selecting stage to build
25. Parallel build 25 
26. New build tool: BuildKit 26
27. Docker 18.06 (July 2018) default since Docker 23.0 27
28. DOCKER_BUILDKIT=1 docker build … docker buildx build … 28
29. Multi-stage in Python FROM python:3.12-slim - bookworm AS builder RUN apt - get update & & apt - get install - y \ default - libmysqlclient - dev build - essential pkg - conf i g \ # . . . RUN python - m venv /opt/venv & & . /opt/venv/bin/activate \ & & pip install - - no - cache - dir - r requirements.txt FROM python:3.12-slim - bookworm AS production RUN apt - get update & & apt - get install - y \ libmariadb3 \ # . . . COPY - - from=builder /opt/venv /opt/venv # . . . 29 0 upgraded, 3 newly installed, … Need to get 193 kB of archives. After this operation, 886 kB of…
30. virtualenv in Docker? 30
31. Makes copying installed dependencies much easier 31
32. Running pip as root is not a good idea 32
33. Activating the virtualenv COPY - - from=builder /opt/venv /opt/venv ENV VIRTUAL_ENV="/opt/venv" \ PATH="/opt/venv/bin:$PATH" 33
34. 34 FROM base AS production FROM python:3.12-slim - bookworm AS base
37.   Caching 37  
38. Inline Cache: min mode 38
39. docker buildx build - - push - t <image> \ - - cache - to type=inline \ - - cache - from type=registry,ref=<prev - image> \ . 39 Use inline cache
40. Registry Cache: min or max mode 40
41. docker buildx build - - push - t <image> \ - - cache - to type=registry,ref=<cache - image>,mode=max \ - - cache - from type=registry,ref=<prev - cache - image> \ . 41 Use registry cache
42. - - cache - to type=…,compression=zstd, compression - level=3 42 Cache compression
43. - - cache - to type=gha - - cache - from type=gha 43 GitHub Actions Cache
44.  RUN mounts 44 
45. bind mount 45 RUN \ - - mount=type=bind,source=r.txt,target=/tmp/r.txt \ pip install - r /tmp/r.txt
46. Good for large source files 46
47. 47 RUN \ - - mount=type=bind,from=pdm,source=/pdm,target=/pdm \ /pdm/bin/pdm install . . .
48. cache mount 48 RUN - - mount=type=cache,target=/tmp/pip - cache \ pip install - - cache - dir=/tmp/pip - cache …
49. Stays on the build machine 49 Work-around for GitHub Actions: https://github.com/reproducible-containers/buildkit-cache-dance
50. secret mount RUN \ - - mount=type=secret,id=pip,target=/home/user/.netrc,\ uid=1000,gid=1000,mode=0400 \ pip install - r requirements.txt $ docker buildx build - - secret id=pip,src=.netrc . 50
51. Private package index, AWS credentials, etc. 51
52.  Non-privileged user 52 
53. RUN groupadd - - gid 1000 app \ & & useradd - - create - home - - uid 1000 - - gid app app USER app WORKDIR /home/app/example - app 53 Change user in Docker image
54. COPY - - chown=app:app . /home/app/example - app COPY - - chmod=700 ./run.sh /home/app/run.sh 54 Change owner/permissions
55. Security updates 55 
56. Running upgrade RUN apt - get update \ & & apt - get upgrade - y \ & & rm - rf /var/lib/apt/lists / * 56
57. Update as last step FROM python:3.12-slim - bookworm AS production # . . . USER root RUN apt - get update & & apt - get upgrade \ & & rm - rf /var/lib/apt/lists / * USER app CMD ["gunicorn", "wsgi:app"] 57
58. Update as the first step FROM python:3.12-slim - bookworm AS production RUN apt - get update & & apt - get upgrade & & apt - get install - y \ libmariadb3 \ & & rm - rf /var/lib/apt/lists / * \ # . . . 58
59. docker buildx build - - no - cache . . . 59 Build without cache
60. docker run - - rm \ - - user root \ - - entrypoint /bin/bash \ <image> \ - c 'apt - get update - qq \ & & ( \ apt - get upgrade - - dry - run \ | grep -E "\\(.* (Debian-Security:|Ubuntu:[^/] * / [^-]* - security)" \ )' 60 Check your image for security updates
61. Conclusion 61 
62. 62 Dockerfile Examples Poetry pip uv https://github.com/jsmitka/examples-python-images-production
63. Caching • MIN or MAX mode? • Cache provider: • Inline cache • Registry cache + compression • GitHub Actions (gha) 63
64. Regular re-builds 64
65.   Every project is di ff erent 65   
66. Thanks for attention! 66 @jansmitka @jansmitka@fosstodon.org www.linkedin.com/in/jansmitka https://lynt.cz/