How Python manages the memory?

Photo by Chris Ried on Unsplash

Python is a versatile, beginner-friendly language, and part of its charm lies in how it handles memory management seamlessly. Python’s memory management is automatic, which means developers can focus on solving problems rather than worrying about allocating or deallocating memory manually.

In this blog, we’ll explore how Python manages memory, illustrate it with examples. By the end, you’ll understand how Python handles memory like a pro and how you can write memory-efficient code. There are some witty comments which can help you track if the blog data is flowing in your memory or overflowing.

Python uses a dynamic and automatic memory management system. This system is responsible for:

1. Memory Allocation: Allocating memory to objects.

2. Memory Deallocation: Releasing memory for objects that are no longer in use.

3. Garbage Collection: Cleaning up unused objects automatically.

But how does Python do all of this? Let’s explore step by step.

The Memory Allocation Process

When you create a variable in Python, you're not directly creating space for the variable in memory. Instead, you're creating a reference to an object in memory. For instance:

x = 10
y = 10
print(id(x), id(y))  #id function return object's memory address

Output:

140726785018584 140726785018584

Python optimizes memory usage by reusing immutable objects like integers and strings. In this case, x and y point to the same memory location because 10 is immutable.

Why don’t Python developers argue about memory allocation?
Because references speak for themselves!

Garbage Collection: The Cleanup Crew

Python’s garbage collector (GC) uses reference counting to track objects. If the reference count of an object drops to zero, it’s no longer needed, and the GC sweeps it away.

Let’s see it in action:

import gc
class Demo:
    def __del__(self):
        print("Garbage collector is reclaiming memory!")
# Create an object
def my_func():
  obj = Demo()
  print("Object created")

my_func()
print("Function Ended")

Output:

Object created
Garbage collector is reclaiming memory!
Function Ended

Garbage collection makes Python highly efficient, but sometimes, circular references (e.g., objects referring to each other) can complicate things. Python also employs generational garbage collection. This advanced method is designed to handle cyclic references and improve memory management efficiency.

What is garbage collectors favourite type of variable?

The one with no strings attached

Mutable vs Immutable Objects

Understanding the difference between mutable and immutable objects is key to grasping Python’s memory behavior.

Immutable Example

Immutable objects (e.g., integers, strings, tuples) cannot be changed in place. Instead, a new object is created when modified.

a = "hello"
b = a
a = "world" # Creates a new object
print(b) # Output: hello

Mutable Example

Mutable objects (e.g., lists, dictionaries) can be changed in place, so the memory reference stays the same.

lst1 = [1, 2, 3]
lst2 = lst1
lst1.append(4) # Modifies the original object
print(lst2) # Output: [1, 2, 3, 4]

Python devs: "Immutable means immutable."
Mutable objects: "Hold my reference."

The Role of the Python Memory Manager

Python splits memory into:

Heap Memory: Python objects and data structures live in a private heap, managed internally by the Python Memory Manager and garbage collector. Developers don’t have direct access to this heap but can use the sys module to monitor memory usage.

Stack Memory: Handles function calls and local variables. Memory in stack is temporary and automatically deallocates once function call ends.

Tips to Write Memory-Efficient Python Code

1. Use Generators Instead of Lists:
Generators save memory by yielding items one at a time rather than storing them all in memory.

def generator_example(n):
    for i in range(n):
        yield i
gen = generator_example(10)
print(next(gen))  # 0

2. Avoid Creating Unnecessary References:
Large datasets? Use slicing and iterators wisely to avoid holding unnecessary data.

# Avoid this
large_data = [i for i in range(10**6)]
processed_data = large_data
print(len(processed_data))

In this case even if large_data is no longer needed, ther reference processed_data prevents the list from being garbage collected. We should ideally explicitly del or assign the large_data to None when it is no longer needed.

3. Be Mindful of Mutable Defaults:
Default mutable arguments in functions can cause surprises.

def create_list(val, lst=[]):
    lst.append(val)
    return lst
print(create_list(1))  # [1]
print(create_list(2))  # [1, 2] - Oops!
# 2 got appended in the same list as lst is mutable

Conclusion

Python makes memory management easy and intuitive for developers. Its automatic garbage collection, combined with the principles of immutability and efficient allocation, allows developers to focus on their code rather than worrying about memory leaks. But it is always advisable to follow best memory practices while coding for a production ready, bug free code.

Happy coding, and may your references always point to greatness!

In Plain English 🚀

Thank you for being a part of the In Plain English community! Before you go:

• Be sure to clap and follow the writer ️👏️️
• Follow us: X | LinkedIn | YouTube | Discord | Newsletter | Podcast
• Create a free AI-powered blog on Differ.
• More content at PlainEnglish.io