Playback speed
Share post
Share post at current time

Recording: CPython and ELF Essentials for Building a Basic Remote Profiler

Yesterday, we did the live session on the internals of remote sampling profilers. We learned the internals that are required to build such tools. Building these tools is probably one of the most interesting systems programming projects that you can do to not only learn the internals of a programming language, but also learn the ELF file format.

Specifically, a remote sampling profiler for a language like Python is able to attach to a running Python process. It is then able to read the memory of that process to figure out what Python code is executing on the interpreter and extract its stack trace.

To be able to do this, three pieces are required:

  • Ability to read the memory of an external process

  • Knowledge of the Python runtime data structures that you need to traverse to extract the stack trace once you are able to read the memory of the Python process.

  • Figuring out the memory addresses of these data structures in the memory of the running Python process. This is the most tricky and interesting part. It requires parsing the python executable binary file to read the memory address that the linker has specified where the runtime data structure should be loaded. And it also requires understanding of how the program loader loads the executable into memory.

We covered all of this yesterday. It was quite a bit of detail to go through but worth it!

You can also access the slides at the below link:


This post is for paid subscribers

Confessions of a Code Addict
Confessions of a Code Addict Podcast
Deep dives into varied topics on Computer Science including compilers, programming languages, database internals, AI and more. Subscribe for insights and advance your engineering skills!