Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Python Memory Management

Python Memory Management

A simple talk about simple problems with Python memory allocator

Tomek Paczkowski

November 28, 2013

More Decks by Tomek Paczkowski

Other Decks in Programming


  1. Case Study • Long lived web process • Periodically allocates

    boatloads of memory • Never releases it
  2. Fun with Python allocator • Python does not use malloc

    directly — too costly for small objects • Instead implements more sophisticated allocator on top of malloc
  3. Free lists • For handful of most common types Python

    keeps unused objects in so called free lists • Those are most significantly: lists, dictionaries, frames • Speeds up code execution immensely
  4. Conclusions • Sometimes memory leak is not what it seems

    • glibc malloc is not the best of breed • do memory intensive work in subprocess • be mindful when using C extensions