Last weekend, we had a live session on the internals of memory management in CPython. Unfortunately, the recording of that session got corrupted, so I recorded a video on it.
CPython uses a combination of reference counting and a generational garbage collector (GC) to manage the memory of the Python programs. Reference counting is a fairly simple technique and I’ve written a very detailed article on it. So in this video, I give a basic overview of reference counting and its implementation, but the majority of the focus is on the garbage collector and its implementation.
The garbage collector in CPython is used for cleaning up memory of objects which have cyclic references and thus cannot be cleaned up reference counting.
The video covers the following topics:
How CPython manages memory
Reference counting & its implementation details in CPython at the interpreter level
In what situations reference counting does not work?
GC implementation details:
What is a generational GC?
How and when the GC is scheduled to run?
What does the GC do in CPython?
The cyclic reference detection algorithm in the CPython GC
Changes in reference counting and GC implementation due to removal of the global interpreter lock (GIL)
Upcoming Article
Although, this video is very detailed and tells you everything you need to know about theGC in CPython, it does not go into the source code (I do show some important struct definitions).
I’ve a very detailed article coming up in a few days on the GC internals in CPython, which goes into all the gory implementation details and walks you through the code to show how everything happens. So watch out for that as well. Perhaps having watched this video will help you get a better perspective when reading that article.
Upcoming Live Session on Building a Basic Remote Profiler for Python
I also have another live session coming up on 26th May where we will actually apply our knowledge of the CPython runtime internals to learn how tools like remote profilers or debuggers work. This will be very technical and will have lots of stuff to learn. I’m very excited about. You can checkout the details in the below post.
Live Session: CPython and ELF Essentials for Building a Basic Remote Profiler
Support Confessions of a Code Addict
If you enjoy reading my articles and find value in them, please consider supporting Confessions of a Code Addict by becoming a paid subscriber. It takes several days of work to produce every piece that I write and your support will help keep it going. Although, I try to keep most of my articles open, I offer the following additional perks to the paid supporters:
Free access to the monthly live sessions
Access to past recordings of all the live sessions
Option to schedule a 30 minute meeting with me every month to discuss anything you wish
Alternative Ways to Support
Many people report that their transactions fail on Substack, so I also have a couple of alternate options.
You can support me on buymeacoffee.com. You can either buy me coffees or become a member and I will upgrade you to a paid subscription here for the equivalent period.
Alternatively, you can also support me through GitHub sponsorship:
