Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Python Memory Management
Search
Tomek Paczkowski
November 28, 2013
Programming
0
67
Python Memory Management
A simple talk about simple problems with Python memory allocator
Tomek Paczkowski
November 28, 2013
Tweet
Share
More Decks by Tomek Paczkowski
See All by Tomek Paczkowski
Dispelling py.test magic
oinopion
0
850
Python Fest lightning talk
oinopion
0
90
Contributing to Django
oinopion
2
560
Django and Time Zones: New in the Trunk
oinopion
4
210
Other Decks in Programming
See All in Programming
アセットのコンパイルについて
ojun9
0
130
Tool Catalog Agent for Bedrock AgentCore Gateway
licux
7
2.5k
FindyにおけるTakumi活用と脆弱性管理のこれから
rvirus0817
0
530
基礎から学ぶ大画面対応(Learning Large-Screen Support from the Ground Up)
tomoya0x00
0
3.3k
個人軟體時代
ethanhuang13
0
330
Navigation 2 を 3 に移行する(予定)ためにやったこと
yokomii
0
340
How Android Uses Data Structures Behind The Scenes
l2hyunwoo
0
480
AIと私たちの学習の変化を考える - Claude Codeの学習モードを例に
azukiazusa1
10
4.4k
Flutter with Dart MCP: All You Need - 박제창 2025 I/O Extended Busan
itsmedreamwalker
0
150
1から理解するWeb Push
dora1998
7
1.9k
Namespace and Its Future
tagomoris
6
710
さようなら Date。 ようこそTemporal! 3年間先行利用して得られた知見の共有
8beeeaaat
3
1.5k
Featured
See All Featured
Git: the NoSQL Database
bkeepers
PRO
431
66k
Faster Mobile Websites
deanohume
309
31k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
What's in a price? How to price your products and services
michaelherold
246
12k
The World Runs on Bad Software
bkeepers
PRO
70
11k
Automating Front-end Workflow
addyosmani
1370
200k
We Have a Design System, Now What?
morganepeng
53
7.8k
Art, The Web, and Tiny UX
lynnandtonic
303
21k
Fantastic passwords and where to find them - at NoRuKo
philnash
52
3.4k
GraphQLとの向き合い方2022年版
quramy
49
14k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
KATA
mclloyd
32
14k
Transcript
Python Memory Management Tomasz Paczkowski Pykonik, 28.11.2013
Case Study • Long lived web process • Periodically allocates
boatloads of memory • Never releases it
Distilled code def main():! big = alloc(100000)! small = alloc(1)!
del big! # memory not released
[Demo time]
Diagnose: Memory Fragmentation big small small big
Solution • Make better use of memory • Subprocess •
jemalloc* via LD_PRELOAD
Fun with Python allocator • Python does not use malloc
directly — too costly for small objects • Instead implements more sophisticated allocator on top of malloc
Pools for integers ints = range(5*1000*1000)! del ints! import gc;
gc.collect(2)
Free lists • For handful of most common types Python
keeps unused objects in so called free lists • Those are most significantly: lists, dictionaries, frames • Speeds up code execution immensely
[Demo time]
Conclusions • Sometimes memory leak is not what it seems
• glibc malloc is not the best of breed • do memory intensive work in subprocess • be mindful when using C extensions
Thanks. Questions?