Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Garbage Collection in Python by Benjamin Peterson
Search
PyCon 2014
April 12, 2014
5
1.4k
Garbage Collection in Python by Benjamin Peterson
PyCon 2014
April 12, 2014
Tweet
Share
More Decks by PyCon 2014
See All by PyCon 2014
Postgres Performance for Humans by Craig Kerstiens
pycon2014
29
3.6k
Technical Onboarding, Training, and Mentoring by Kate Heddleston and Nicole Zuckerman
pycon2014
1
2.3k
"My big gay adventure. Making, releasing and selling an indie game made in python." by Luke Miller
pycon2014
2
1.5k
Farewell and Welcome Home, Python in Two Genders by Naomi_Ceder
pycon2014
1
710
Deliver Your Software in an Envelope by Augie Fackler and Nathaniel Manista
pycon2014
1
520
Hitchhikers Guide to Free and Open Source Participation by Elena Williams
pycon2014
6
1.2k
Localization Revisted (aka. Translations Evolved) by Ruchi Varshney
pycon2014
0
690
Smart Dumpster by Bradley E. Angell
pycon2014
0
490
Software Engineering for Hackers: Bridging the Two Solitudes by Tavish Armstrong
pycon2014
0
710
Featured
See All Featured
No one is an island. Learnings from fostering a developers community.
thoeni
19
3k
Rebuilding a faster, lazier Slack
samanthasiow
79
8.7k
VelocityConf: Rendering Performance Case Studies
addyosmani
326
24k
Unsuck your backbone
ammeep
669
57k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
48k
Six Lessons from altMBA
skipperchong
27
3.5k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
26
1.9k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
95
17k
Building Your Own Lightsaber
phodgson
103
6.1k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
45
2.2k
Keith and Marios Guide to Fast Websites
keithpitt
410
22k
A Tale of Four Properties
chriscoyier
157
23k
Transcript
Garbage Collection Benjamin Peterson
Outline ➔ How GC works in various Python implementations ◆
optimizations ➔ GC semantics subtleties
Part 1 Implementation Basics
Bias & Disclaimer
What is GC in Python? ➔ Unused objects are finalized
and deallocated. ➔ When? ◆ “eventually”... or never! ◆ not running out of memory is good
CPython
CPython: reference counting ➔ Every object has a count of
how many other objects want to keep it alive. ➔ New objects have ref count 1. ➔ When ref count is 0, the object can be deleted.
New object Preexisting objects
Preexisting objects New object
Preexisting objects Dead object
Preexisting objects Dead object Dead object
refcounting example
Reference Counting’s Major Flaw Reference Cycles
Preexisting objects
Preexisting objects Cycle keeps itself alive!
CPython’s cyclic GC ➔ Detects cycles unreachable from the program
and deletes them ➔ Runs every once and while on allocation ➔ on CPython
Preexisting objects Cyclic GC subtracts internal references
Preexisting objects Cycles are now deleted
None
PyPy review ➔ Interpreter written in RPython ➔ RPython translated
to low level language (C) ➔ Interpreter is abstracted from low-level details like GC
PyPy has pluggable GCs
PyPy GC ➔ GC is simply another low-level transform during
translation. ➔ GC algorithm itself is written in RPython. ➔ GC implementation can be selected at translation time. ➔ Current default GC: “minmark”
Mark and Sweep ➔ Step 1: Starting from known live
objects, recursively traverse objects, marking them as reachable. ➔ Step 2: Walk all allocated objects, deleting that ones that aren’t marked as alive. ➔ No need to worry about reference cycles.
Preexisting objects Unreachable object Marking
Preexisting objects Unreachable object Marking GC traverses references to C.
Preexisting objects Unreachable object Sweeping GC notices that D is
unreachable and deletes it.
Fancy PyPy GC Optimizations
Python allocates constantly
“High Infant Mortality”
The nursery ➔ Store newly allocated objects in a “nursery”
➔ Collect the “nursery” often and move surviving objects elsewhere (minor collection) ➔ Garbage collect old objects less frequently (major collection)
GC Pauses ➔ When the GC is running, the program
is not. ➔ Inconvenient for many long running programs like servers. ➔ A deal-breaker for real-time applications like video processing.
PyPy incremental GC ➔ In PyPy 2.2 ➔ Major collection
split into multiple passes, each lasting only a few milliseconds. ➔ “All in all it was relatively painless work.”
GC in PyPy summary ➔ Pluggable ➔ Generational ➔ Incremental
➔ Integrated with the JIT
Part 2 GC Semantics
kills kittens
Cycles with finalizers: a conundrum Which finalizer to run first?
Reference to the alive world.
What to do about cycles with ? ➔ CPython <
3.4: give up ➔ CPython >= 3.4: PEP 442 ➔ PyPy: sort finalizers into a “reasonable” order and run them
PEP 442 - “Safe Object Finalization” ➔ Step 1: Run
finalizers on unreachable cycles (arbitrary order). Resurrect any cycles that become reachable again. ➔ Step 2: Break references in remaining cycles.
Themes ➔ Garbage Collection is hard. ➔ (PyPy’s) GCs are
awesome!
Queries? (
[email protected]
) ➔ PyPy Blog (http://morepypy.blogspot.com) ➔ GC module documentation
➔ Wikipedia article on garbage collection ➔ The source ◆ ◆