Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Python Memory Management
Search
Tomek Paczkowski
November 28, 2013
Programming
0
63
Python Memory Management
A simple talk about simple problems with Python memory allocator
Tomek Paczkowski
November 28, 2013
Tweet
Share
More Decks by Tomek Paczkowski
See All by Tomek Paczkowski
Dispelling py.test magic
oinopion
0
800
Python Fest lightning talk
oinopion
0
89
Contributing to Django
oinopion
2
560
Django and Time Zones: New in the Trunk
oinopion
4
210
Other Decks in Programming
See All in Programming
チームのテスト力を総合的に鍛えて品質、スピード、レジリエンスを共立させる/Testing approach that improves quality, speed, and resilience
goyoki
5
940
システム成長を止めない!本番無停止テーブル移行の全貌
sakawe_ee
1
210
AI Agent 時代のソフトウェア開発を支える AWS Cloud Development Kit (CDK)
konokenj
3
280
生成AI時代のコンポーネントライブラリの作り方
touyou
1
250
Modern Angular with Signals and Signal Store:New Rules for Your Architecture @enterJS Advanced Angular Day 2025
manfredsteyer
PRO
0
230
MCPを使ってイベントソーシングのAIコーディングを効率化する / Streamlining Event Sourcing AI Coding with MCP
tomohisa
0
120
AIと”コードの評価関数”を共有する / Share the "code evaluation function" with AI
euglena1215
1
170
Systèmes distribués, pour le meilleur et pour le pire - BreizhCamp 2025 - Conférence
slecache
0
120
おやつのお供はお決まりですか?@WWDC25 Recap -Japan-\(region).swift
shingangan
0
140
GitHub Copilot and GitHub Codespaces Hands-on
ymd65536
2
150
ruby.wasmで多人数リアルタイム通信ゲームを作ろう
lnit
3
490
Claude Code + Container Use と Cursor で作る ローカル並列開発環境のススメ / ccc local dev
kaelaela
10
6k
Featured
See All Featured
How GitHub (no longer) Works
holman
314
140k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
35
2.4k
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.9k
Building an army of robots
kneath
306
45k
The Cost Of JavaScript in 2023
addyosmani
51
8.5k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Mobile First: as difficult as doing things right
swwweet
223
9.7k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
281
13k
Bash Introduction
62gerente
613
210k
Rebuilding a faster, lazier Slack
samanthasiow
83
9.1k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
48
2.9k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Transcript
Python Memory Management Tomasz Paczkowski Pykonik, 28.11.2013
Case Study • Long lived web process • Periodically allocates
boatloads of memory • Never releases it
Distilled code def main():! big = alloc(100000)! small = alloc(1)!
del big! # memory not released
[Demo time]
Diagnose: Memory Fragmentation big small small big
Solution • Make better use of memory • Subprocess •
jemalloc* via LD_PRELOAD
Fun with Python allocator • Python does not use malloc
directly — too costly for small objects • Instead implements more sophisticated allocator on top of malloc
Pools for integers ints = range(5*1000*1000)! del ints! import gc;
gc.collect(2)
Free lists • For handful of most common types Python
keeps unused objects in so called free lists • Those are most significantly: lists, dictionaries, frames • Speeds up code execution immensely
[Demo time]
Conclusions • Sometimes memory leak is not what it seems
• glibc malloc is not the best of breed • do memory intensive work in subprocess • be mindful when using C extensions
Thanks. Questions?