Slide 1
Slide 1 text
José Manuel Ortega
@jmortegac
Python Memory
Management 101
Deeping in Garbage collector
Slide 2
Slide 2 text
About me
● @jmortegac
● http://jmortega.github.io
● https://www.linkedin.com/in/jmortega1/
Slide 3
Slide 3 text
Agenda
● Introduction to memory management
● Garbage collector and reference counting
with python
● Review the gc module for configuring the
python garbage collector
● Best practices for memory managment
Slide 4
Slide 4 text
Introduction to memory management
● Memory management is the process of
efficiently allocating, de-allocating, and
coordinating memory so that all the
different processes run smoothly and can
optimally access different system
resources.
Slide 5
Slide 5 text
Python Objects in Memory
Slide 6
Slide 6 text
Python Memory Manager
Slide 7
Slide 7 text
Heap allocation
Slide 8
Slide 8 text
Heap allocation
def main():
x=300
print(id(x))
w=fun(x)
print(id(w))
def sqr(x):
print (id(x))
z=x*x
print(id(z))
return z
if __name__ == "__main__":
main()
Slide 9
Slide 9 text
Python Objects in Memory
● Each variable in Python acts as an object
● Python is a dynamically typed language
which means that we do not need to
declare types for variables.
Slide 10
Slide 10 text
Python Objects in Memory
Slide 11
Slide 11 text
Python Objects in Memory
Slide 12
Slide 12 text
Python Objects in Memory
Slide 13
Slide 13 text
Python Objects in Memory
Slide 14
Slide 14 text
id() method
Slide 15
Slide 15 text
id() function
Slide 16
Slide 16 text
id() function
Slide 17
Slide 17 text
Is Operator
Slide 18
Slide 18 text
Reference counting
● Python manages objects by using reference
counting
● Reference counting works by counting the
number of times an object is referenced by
other objects in the application.
● When references to an object are removed,
the reference count for an object is
decremented.
Slide 19
Slide 19 text
Reference counting
● A reference is a container object pointing at
another object.
● Reference counting is a simple technique
in which objects are allocated when there is
reference to them in a program
Slide 20
Slide 20 text
Reference counting
● when reference count increases?
○ x=1
○ def(x):
○ list.append(x)
Slide 21
Slide 21 text
Reference counting
Slide 22
Slide 22 text
Reference counting
Slide 23
Slide 23 text
Reference counting
Slide 24
Slide 24 text
Reference counting
● Easy to implement
● Objects are immediately deleted when
reference counter is 0
✗ Not thread-safe
✗ Doesn’t detect cyclic references
✗ space overhead - reference count is
stored for every object
Slide 25
Slide 25 text
Python Garbage collector
● The Python Garbage Collector (GC) runs
during the program execution and is
triggered if the reference count reduces to
zero.
Slide 26
Slide 26 text
Garbage collector(GC) module
Slide 27
Slide 27 text
Python Garbage collector
● Reference Counting + Generational GC
● RefCount reaches zero, immediate
deletion
● Deleted objects with cyclic references
are deleted with Tracing GC
Slide 28
Slide 28 text
Garbage collector(GC) reference cycle
Slide 29
Slide 29 text
Garbage collector(GC) reference cycle
>>> def ref_cycle():
... list = [1, 2, 3, 4]
... list.append(list)
... return list
Slide 30
Slide 30 text
Garbage collector(GC) reference cycle
import gc
for i in range(8):
ref_cycle()
n = gc.collect()
print("Number of unreachable objects collected by GC:", n)
print("Uncollectable garbage:", gc.garbage)
print("Number of unreachable objects collected by GC:",
gc.collect())
Slide 31
Slide 31 text
Garbage collector(GC) reference cycle
Slide 32
Slide 32 text
Python Object Graphs
https://mg.pov.lt/objgraph/
>>> x = []
>>> list = [x, [x], dict(x=x)]
>>> import objgraph
>>> objgraph.show_refs([y],
filename='sample-graph.png')
Slide 33
Slide 33 text
Best practices for memory managment
● Using gc.collect() carefully
print("Collecting...")
n = gc.collect()
print("Number of unreachable objects collected:", n)
print("Uncollectable garbage:", gc.garbage)
Slide 34
Slide 34 text
Best practices for memory managment
● Using with context manager for working
with files
with open('data.txt', 'r') as file:
data = ','.join(line.strip() for line in file)
Slide 35
Slide 35 text
Best practices for memory managment
● Avoid List Slicing with [:]
list= [1,2,3,4]
list[1:3]
list[slice(1,3)]
Slide 36
Slide 36 text
Best practices for memory managment
● String Concatenation
string= “hello”
string+= “world”
wordList = ("hello", "world")
string = " ".join(wordList)
Slide 37
Slide 37 text
Best practices for memory managment
● Use Iterators and Generators
Slide 38
Slide 38 text
References
● https://stackabuse.com/basics-of-memory-management-in-p
ython/
● https://realpython.com/python-memory-management
● https://rushter.com/blog/python-garbage-collector
● https://pythonchb.github.io/PythonTopics/weak_references.
html
Slide 39
Slide 39 text
https://www.youtube.com/c/JoseManuelOrtegadev
Slide 40
Slide 40 text
https://speakerdeck.com/jmortega