Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Python Memory Management
Search
Tomek Paczkowski
November 28, 2013
Programming
0
62
Python Memory Management
A simple talk about simple problems with Python memory allocator
Tomek Paczkowski
November 28, 2013
Tweet
Share
More Decks by Tomek Paczkowski
See All by Tomek Paczkowski
Dispelling py.test magic
oinopion
0
800
Python Fest lightning talk
oinopion
0
88
Contributing to Django
oinopion
2
560
Django and Time Zones: New in the Trunk
oinopion
4
210
Other Decks in Programming
See All in Programming
Startups on Rails in Past, Present and Future–Irina Nazarova, RailsConf 2025
irinanazarova
0
100
ISUCON研修おかわり会 講義スライド
arfes0e2b3c
1
440
Python型ヒント完全ガイド 初心者でも分かる、現代的で実践的な使い方
mickey_kubo
1
110
初学者でも今すぐできる、Claude Codeの生産性を10倍上げるTips
s4yuba
16
11k
XP, Testing and ninja testing
m_seki
3
240
Google Agent Development Kit でLINE Botを作ってみた
ymd65536
2
250
明示と暗黙 ー PHPとGoの インターフェイスの違いを知る
shimabox
2
510
git worktree × Claude Code × MCP ~生成AI時代の並列開発フロー~
hisuzuya
1
570
PicoRuby on Rails
makicamel
2
130
#QiitaBash MCPのセキュリティ
ryosukedtomita
1
1.3k
AIエージェントはこう育てる - GitHub Copilot Agentとチームの共進化サイクル
koboriakira
0
590
チームで開発し事業を加速するための"良い"設計の考え方 @ サポーターズCoLab 2025-07-08
agatan
1
420
Featured
See All Featured
How GitHub (no longer) Works
holman
314
140k
The World Runs on Bad Software
bkeepers
PRO
69
11k
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.9k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
130
19k
Unsuck your backbone
ammeep
671
58k
Why Our Code Smells
bkeepers
PRO
336
57k
Making Projects Easy
brettharned
116
6.3k
Building Adaptive Systems
keathley
43
2.7k
The Invisible Side of Design
smashingmag
301
51k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
How STYLIGHT went responsive
nonsquared
100
5.6k
Building an army of robots
kneath
306
45k
Transcript
Python Memory Management Tomasz Paczkowski Pykonik, 28.11.2013
Case Study • Long lived web process • Periodically allocates
boatloads of memory • Never releases it
Distilled code def main():! big = alloc(100000)! small = alloc(1)!
del big! # memory not released
[Demo time]
Diagnose: Memory Fragmentation big small small big
Solution • Make better use of memory • Subprocess •
jemalloc* via LD_PRELOAD
Fun with Python allocator • Python does not use malloc
directly — too costly for small objects • Instead implements more sophisticated allocator on top of malloc
Pools for integers ints = range(5*1000*1000)! del ints! import gc;
gc.collect(2)
Free lists • For handful of most common types Python
keeps unused objects in so called free lists • Those are most significantly: lists, dictionaries, frames • Speeds up code execution immensely
[Demo time]
Conclusions • Sometimes memory leak is not what it seems
• glibc malloc is not the best of breed • do memory intensive work in subprocess • be mindful when using C extensions
Thanks. Questions?