Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Python Memory Management

Python Memory Management

A simple talk about simple problems with Python memory allocator

Avatar for Tomek Paczkowski

Tomek Paczkowski

November 28, 2013
Tweet

More Decks by Tomek Paczkowski

Other Decks in Programming

Transcript

  1. Case Study • Long lived web process • Periodically allocates

    boatloads of memory • Never releases it
  2. Fun with Python allocator • Python does not use malloc

    directly — too costly for small objects • Instead implements more sophisticated allocator on top of malloc
  3. Free lists • For handful of most common types Python

    keeps unused objects in so called free lists • Those are most significantly: lists, dictionaries, frames • Speeds up code execution immensely
  4. Conclusions • Sometimes memory leak is not what it seems

    • glibc malloc is not the best of breed • do memory intensive work in subprocess • be mindful when using C extensions