So, the question arises: how do frameworks like PyTorch generate random numbers in parallel on GPU devices? Because if random number generation becomes a bottleneck, it can significantly slow down the entire training or inference pipeline.

The answer lies in a clever algorithm called Philox, a counter-based parallel random number generator. In this article, we’ll explore:

1. Why traditional random number generators don’t parallelize well

2. How Philox works and what makes it different

3. How to parallelize random number generation using Philox

4. PyTorch’s implementation of Philox by dissecting its C++ and CUDA code

By the end, you’ll understand how that simple torch.randn() call efficiently generates millions of random numbers in parallel on your GPU while maintaining perfect reproducibility.

Cut Code Review Time & Bugs in Half (Sponsored)

Code reviews are critical but time-consuming. CodeRabbit acts as your AI co-pilot, providing instant Code review comments and potential impacts of every pull request.

Beyond just flagging issues, CodeRabbit provides one-click fix suggestions and lets you define custom code quality rules using AST Grep patterns, catching subtle issues that traditional static analysis tools might miss.

CodeRabbit has so far reviewed more than 10 million PRs, installed on 2 million repositories, and used by 100 thousand Open-source projects. CodeRabbit is free for all open-source repo’s.

Get Started Today

Problem with Traditional PRNGs

Let’s start by developing an intuition about why traditional pseudo random number generators (PRNGs) are sequential and not suitable for parallel hardware, such as GPUs.

A PRNG needs to be able to reproduce the same sequence of random numbers when initialized with a specific seed. A natural way of achieving this is through a state transformation function that takes the current state of the generator as input and produces a new state. As long as the function is deterministic, it is guaranteed that we can reproduce the exact same sequence of numbers starting from the same initial state. Mathematically, it can be expressed like this:

Here, the next state is derived by applying the function f on the current state s_n. As you can see, this is a sequential model where you can’t jump ahead arbitrarily without computing all the previous states, and you can’t shard the generation of the random numbers by distributing the work across threads.

To parallelize the generation of random numbers, we need a different model where we can directly generate the nth random number without having to go through the generation of all the previous n-1 numbers. Mathematically, it should look like this:

Where x_n is the nth random number we wish to generate by applying a function b. Here, we can think of the input n as an integer counter and as such the PRNGs that follow this model are called counter-based random number generators. One such counter-based PRNG is the Philox PRNG, used widely in frameworks such as PyTorch for parallel random number generation on GPUs.

Let’s understand how Philox works.

How Philox Works

The Philox algorithm, short for “Product HI, LOw, with XOR”, is a counter-based PRNG that was designed specifically for parallel computation. It was introduced by Salmon et al. in 2011 as part of the Random123 library. The key insight behind Philox is that we can use a cryptographic-like construction to transform a counter into a pseudorandom number.

The Core Idea: Treating RNG as Encryption

We can think of the counter-based RNG problem this way: we want to take a sequence of integers (0, 1, 2, 3, …) and scramble them so thoroughly that they appear random. This is conceptually similar to what a block cipher does in cryptography, it takes a plaintext message and a key, then produces a ciphertext that looks random.

In Philox’s case:

• The counter (n) acts like the plaintext

• The seed acts like the encryption key

• The output is our pseudorandom number

The beauty of this approach is that any thread can independently compute its random number by knowing just two things: which counter value it needs (its position in the sequence) and the seed. No synchronization or communication with other threads is needed.

The Philox Construction

Philox operates on fixed-size inputs and outputs. The most common variant is Philox-4x32, which means:

• 4: Works with 4 32-bit integers at a time

• 32: Each integer is 32 bits wide

So Philox-4x32 takes a 128-bit counter (represented as four 32-bit integers) and produces a 128-bit output (four 32-bit random numbers). This is perfect for generating multiple random numbers at once, which is common in GPU workloads.

The algorithm consists of applying multiple rounds of a transformation function. Each round performs these operations:

1. Multiplication and splitting: Multiply pairs of the input integers and split the results into high and low parts

2. XOR with keys: XOR certain parts with key-derived values

3. Permutation: Shuffle the positions of the integers

Let’s break down a single round in detail. Philox-4x32 works with four 32-bit values, which we’ll call (c0​,c1​,c2​,c3​). Each round transforms these values through the following steps:

Step 1: Multiply and Split

Take the first pair (c0​,c1​) and the second pair (c2​,c3​). Multiply each by a carefully chosen constant:

For Philox-4x32, these constants are:

• M0​=0xD2511F53

• M1​=0xCD9E8D57

These constants were chosen through careful analysis to ensure good statistical properties. When we multiply two 32-bit numbers, we get a 64-bit result. We split this into:

• High 32 bits: hi(prod)

• Low 32 bits: lo(prod)

Step 2: XOR with Keys

The high parts are XORed with round-specific keys derived from the seed, and with the other input values:

Here, k0​ and k1​ are the key values (derived from the seed), and ⊕ represents the XOR operation.

Step 3: Permutation

Finally, we rearrange the values for the next round. The output of one round becomes:

Notice how the values are shuffled: the low parts of the products go to positions 0 and 2, while the XORed high parts are swapped and go to positions 1 and 3.

Multiple Rounds

To achieve good randomness, Philox-4x32 typically applies 10 rounds of this transformation. After each round except the last, the keys are also updated:

Where w0​=0x9E3779B9 and w1​=0xBB67AE85 are the “Weyl sequence” constants derived from the golden ratio. This ensures that each round uses different key material, increasing the mixing of the input bits.

Visualizing a Complete Philox Transformation

The following diagram shows the complete flow through multiple rounds:

Why This Works

The Philox algorithm achieves good randomness through several mechanisms:

1. Multiplication is a non-linear operation that mixes bits effectively. Small changes in input lead to large changes in output.

2. High-low splitting ensures we use all 64 bits of the multiplication result, not just the lower 32 bits.

3. XOR operations combine different data streams (keys, previous values) in a way that’s invertible but unpredictable without knowing the key.

4. Permutation ensures that the mixing effect propagates to all output positions across rounds.

5. Multiple rounds compound these effects, ensuring that every output bit depends on every input bit in a complex way.

The algorithm has been extensively tested and passes standard statistical tests for randomness like the TestU01 suite, making it suitable for scientific computing and machine learning applications.

Properties of Philox

Before we dive into PyTorch’s implementation, let’s summarize the key properties that make Philox attractive:

• Parallel-friendly: A GPU with thousands of cores can generate thousands of random numbers simultaneously, each using a different counter value.

• Deterministic: Given the same seed and counter, you always get the same output.

• Long period: With a 128-bit counter, you can generate 2^128 random numbers before the sequence repeats numbers, more than enough for any practical application.

• Fast: The operations (multiplication, XOR, bit shifting) are primitive operations that run very efficiently on modern CPUs and GPUs.

• Memory efficient: The generator state is just the counter and key, requiring minimal storage per thread.

Next, let’s understand how Philox can be parallelized.

Parallelizing Philox: Subsequences and Offsets

Now that we understand how the Philox algorithm works, let’s explore what makes it particularly powerful for parallel computing: the ability to generate random numbers across thousands of threads simultaneously without any coordination.

The Random Number Space

Recall that Philox is a counter-based PRNG. At its core, it’s a function that maps a 128-bit counter to a 128-bit random output:

Given a fixed key (derived from the seed), each unique counter value produces a unique set of random numbers. Since we have a 128-bit counter, we have:

Each counter value produces 4 random 32-bit numbers (since 128 bits = 4 × 32 bits), giving us an enormous space of random numbers. We can visualize this as a huge one-dimensional array:

Counter: 0 1 2 3 ... 2^128-1

↓ ↓ ↓ ↓ ↓

Output: [r₀,r₁,r₂,r₃][r₄,r₅,r₆,r₇][r₈,r₉,r₁₀,r₁₁][r₁₂,...]...[...]

How do we partition this massive space across parallel threads? One approach is to split the counter space between the threads.

Partitioning the Counter Space

The key insight is that we can split the 128-bit counter into two parts and use them to create a 2D address space. Think of the counter as having 4 components of 32 bits each: (c0​,c1​,c2​,c3​).

We can partition this as:

• Upper 64 bits: Which thread’s region we’re in

• Lower 64 bits : The position within a thread’s assigned region

This partitioning scheme gives each thread its own “slice” of the random number space:

• Thread 0 gets counters: (∗,∗,0,0) where ∗∗ can be any value

• counter = (0,0,0,0) → first 4 random numbers for thread 0

• counter = (1,0,0,0) → next 4 random numbers for thread 0

• counter = (2,0,0,0) → next 4 random numbers for thread 0

• …

• Thread 1 gets counters: (∗,∗,1,0)

• counter = (0,0,1,0) → first 4 random numbers for thread 1

• counter = (1,0,1,0) → next 4 random numbers for thread 1

• counter = (2,0,1,0) → next 4 random numbers for thread 1

• …

• Thread 2 gets counters: (∗,∗,2,0)

• counter = (0,0,2,0) → first 4 random numbers for thread 2

• And so on…

Terminology: Subsequence and Offset

We now give names to these two parts:

Subsequence: The upper 64 bits of the counter. This identifies which parallel thread or stream we’re referring to. We can have up to 2^64 different subsequences running in parallel.

Offset: The lower 64 bits of the counter. This identifies the position within a subsequence. Each subsequence can generate up to 2^64 sets of random numbers.

Together, they form a coordinate system (s,o) where:

• s is the subsequence (which parallel stream)

• o is the offset (position in that stream)

The total capacity is:

This matches exactly the size of our original counter space, we’ve simply reorganized it into a 2D structure that’s easy to partition across threads.

How Offsets Increment

When a thread generates more random numbers, it increments the offset portion of the counter. Since Philox generates 4 random numbers at once, we typically increment by 1 each time (remembering that each offset value produces 4 numbers):

Thread 0 subsequence = 0:

offset=0: counter=[0,0,0,0] → Philox → [rand₀, rand₁, rand₂, rand₃]

offset=1: counter=[1,0,0,0] → Philox → [rand₄, rand₅, rand₆, rand₇]

offset=2: counter=[2,0,0,0] → Philox → [rand₈, rand₉, rand₁₀, rand₁₁]

...

The offset is really tracking “which batch of 4” we’re on. If we need the 10th random number (index 9, counting from 0):

• Offset = ⌊9/4⌋=2

• Position within batch = 19mod4=1

• So we use counter [2,0,0,0] and take the second output (index 1)

The Power of Skip-Ahead

One powerful consequence of this design is skip-ahead: a thread can jump directly to any offset without computing intermediate values.

Thread 0:

- Jump to offset 1,000,000: counter = [1000000, 0, 0, 0]

- Generate random numbers at this position

- Jump to offset 5,000,000: counter = [5000000, 0, 0, 0]

- No need to compute offsets 1 through 4,999,999!

This is impossible with traditional sequential PRNGs where state n+1n+1 depends on state nn.

Setting Up for PyTorch

Now that we understand how the counter space is partitioned, we can see how PyTorch uses this:

When PyTorch generates random numbers on a GPU:

1. It launches many threads (e.g., 1024 threads)

2. Each thread is assigned a unique subsequence number (typically its thread ID)

3. Each thread starts at offset 0 within its subsequence

4. As each thread generates random numbers, it increments its offset

5. PyTorch tracks the global offset to ensure future operations don’t reuse the same counters

With this foundation, let’s now explore how PyTorch implements these concepts in its Philox engine.

Philox Implementation in PyTorch

PyTorch uses Philox-4x32-10 (4 values of 32 bits, 10 rounds) as its primary PRNG for CUDA operations. The implementation lives in aten/src/ATen/core/PhiloxRNGEngine.h and is designed to work on both CPU and GPU (via CUDA). Let’s dissect this implementation to understand how the theoretical concepts we discussed earlier translate into actual code.

Core Data Structures

The implementation starts by defining some type aliases for clarity:

typedef std::array<uint32_t, 4> UINT4; // Four 32-bit integers

typedef std::array<uint32_t, 2> UINT2; // Two 32-bit integers

typedef std::array<double, 2> DOUBLE2; // Two doubles

typedef std::array<float, 2> FLOAT2; // Two floats

These typedefs make the code more readable. UINT4 represents the 128-bit counter or output (4 × 32 bits = 128 bits), while UINT2 represents the 64-bit key (2 × 32 bits = 64 bits).

The PhiloxEngine Class Structure

The philox_engine class maintains four critical pieces of state:

private:

detail::UINT4 counter_; // 128-bit counter (c₀, c₁, c₂, c₃)
detail::UINT4 output_; // Cached output from last round
detail::UINT2 key_; // 64-bit key derived from seed (k₀, k₁)
uint32_t STATE; // Position in current output (0-3)

Let’s understand each field:

counter_: This is the 128-bit counter that gets incremented and transformed through the Philox rounds. It’s divided into four 32-bit components:

• counter_[0] and counter_[1]: Lower 64 bits represent the offset (which random number in the subsequence)

• counter_[2] and counter_[3]: Upper 64 bits represent the subsequence (which parallel stream)

key_: The 64-bit key derived from the seed. This remains constant for a given seed and is used in the XOR operations during each round.

output_: Philox generates 4 random 32-bit numbers at once. This field caches those numbers so we don’t have to recompute them for every call.

STATE: A simple counter (0-3) that tracks which of the four cached output values to return next. This is an optimization to avoid regenerating when we have unused random numbers.

Initialization and State Management

The constructor initializes the engine with a seed, subsequence, and offset:

The C10_HOST_DEVICE macro is crucial here, it tells the compiler that this function can run on both the CPU (host) and GPU (device). This allows the same code to be used in both contexts.

Let’s look at how reset_state sets up the initial state:

This initialization strategy is clever:

1. The seed is split into the two key components key_[0] and key_[1]

2. The subsequence goes into the upper half of the counter (counter_[2] and counter_[3])

3. The offset (lower half of counter) starts at zero but can be set later via incr_n(offset)

This design allows for massive parallelism. Imagine running 1024 CUDA threads simultaneously:

Thread 0: subsequence=0, offset=0 → counter = [0, 0, 0, 0]

Thread 1: subsequence=1, offset=0 → counter = [0, 0, 1, 0]

Thread 2: subsequence=2, offset=0 → counter = [0, 0, 2, 0]

...

Thread 1023: subsequence=1023, offset=0 → counter = [0, 0, 1023, 0]

Each thread has a unique counter value from the start, so they all generate independent random sequences without any coordination.

The Core Algorithm: Single Round

Now let’s examine the heart of the Philox algorithm—the single_round function:

Let’s break this down step by step, mapping it to our earlier theoretical description:

Step 1: Multiply and Split

uint32_t lo0 = mulhilo32(kPhiloxSA, ctr[0], &hi0);
uint32_t lo1 = mulhilo32(kPhiloxSB, ctr[2], &hi1);

Here we multiply:

• ctr[0] by kPhiloxSA (the constant 0xD2511F53)

• ctr[2] by kPhiloxSB (the constant 0xCD9E8D57)

The mulhilo32 function performs the multiplication and splits the 64-bit result:

• Returns the low 32 bits (lo0 or lo1)

• Stores the high 32 bits in the passed pointer (hi0 or hi1)

Let’s look at mulhilo32 itself:

This function has two implementations:

On CUDA (GPU): Uses the intrinsic __umulhi which directly computes the high 32 bits of a multiplication. This is extremely fast on GPU hardware.

On CPU: Promotes both operands to 64 bits, multiplies them, then extracts high and low parts manually via shifting and casting.

Here’s what happens mathematically:

Step 2: XOR and Permute

ret[0] = hi1 ^ ctr[1] ^ in_key[0];
ret[1] = lo1;
ret[2] = hi0 ^ ctr[3] ^ in_key[1];
ret[3] = lo0;

Notice the pattern:

• ret[0]: Takes hi1 (high bits from second multiplication), XORs with ctr[1] and in_key[0]

• ret[1]: Simply uses lo1 (low bits from second multiplication)

• ret[2]: Takes hi0 (high bits from first multiplication), XORs with ctr[3] and in_key[1]

• ret[3]: Simply uses lo0 (low bits from first multiplication)

Let us visualize this transformation:

This permutation ensures that bits from different positions get mixed together in subsequent rounds.

Constants: The Magic Numbers

You might wonder where these constants come from:

static const uint32_t kPhilox10A = 0x9E3779B9; // Weyl sequence
static const uint32_t kPhilox10B = 0xBB67AE85; // Weyl sequence
static const uint32_t kPhiloxSA = 0xD2511F53; // Multiplier
static const uint32_t kPhiloxSB = 0xCD9E8D57; // Multiplier

Weyl sequence constants (kPhilox10A and kPhilox10B): These are derived from the golden ratio. The constants are:

The golden ratio has special properties that make it useful for distributing values uniformly. These constants are added to the key after each round to ensure different key material is used.

Multiplier constants (kPhiloxSA and kPhiloxSB): These were carefully chosen through empirical testing to maximize statistical quality. They need to have good bit-mixing properties when multiplied with typical counter values.

Running Multiple Rounds

The rand function orchestrates running all rounds:

This is straightforward:

1. Run n_rounds - 1 iterations where we:

   1. Apply single_round to transform the counter

   2. Update the key by adding the Weyl constants

2. Apply one final round without updating the key

By default, PyTorch uses 10 rounds (n_rounds = 10), which provides a good balance between performance and statistical quality.

Generating Random Numbers: The Operator

The operator () is what users call to get random numbers:

This function is clever in its efficiency:

Check if we need new random numbers: if(STATE == 0) checks if we’ve exhausted the previous batch. Remember, STATE cycles through 0, 1, 2, 3.

Generate a batch: When needed, it:

• Runs the full Philox algorithm via rand(counter, key, n_rounds)

• Stores the result in output_ (four 32-bit random numbers)

• Increments the counter for next time via incr()

Return next value: Grab the current position from output_, then advance STATE.

The line STATE = (STATE + 1) & 3 is a bit trick equivalent to STATE = (STATE + 1) % 4, using bitwise AND since 3 is binary 11.

This batching strategy is a significant performance optimization. Instead of running Philox for every random number, we run it once per four random numbers.

Counter Increment Logic

The counter increment operations deserve special attention because they handle the 128-bit arithmetic correctly. Let’s start with the simple case:

This increments the 128-bit counter by 1. The logic is:

1. Increment counter_[0] (least significant 32 bits)

2. If it’s non-zero after increment, we’re done (no overflow)

3. If it overflowed to zero, carry to counter_[1]

4. Continue propagating carries until we find a non-zero result

The more complex function is incr_n, which increments by an arbitrary 64-bit value:

This function is more intricate because it needs to:

1. Split the 64-bit increment n into nlo and nhi

2. Add nlo to counter_[0]

3. Detect overflow by checking if counter_[0] < nlo (if the result is less than what we added, overflow occurred)

4. If overflow, increment nhi to carry over

5. Add nhi to counter_[1] and check for overflow again

6. If still overflowing, propagate to the upper 64 bits

The overflow detection counter_[0] < nlo is a standard technique in multi-precision arithmetic. After adding, if the result is less than one of the operands, an overflow must have occurred since we’re working with unsigned integers.

Converting to Floating Point

For machine learning applications, we often need floating-point random numbers in the range [0, 1), while Philox gives us integers. So, PyTorch applies a conversion function:

This function is carefully designed:

Mask off sign bit: value & 0x7FFFFFFF clears the highest bit, giving us values from 0 to 2^31−1

Scale down: Multiplying by scale = 4.6566127342e-10 maps these integers to floats in [0, 1).

The scale factor is calculated as:

Why use only 31 bits instead of all 32? Because:

1. We want only positive values (for [0, 1) range)

2. The highest representable float less than 1.0 needs careful handling

3. Using 31 bits avoids potential rounding issues near 1.0

Normal Distribution Generation

The randn function generates normally distributed random numbers using the Box-Muller transform:

The Box-Muller transform converts two uniform random variables U1​,U2​∼Uniform(0,1) into a normal random variable Z∼N(0,1):

Memory Layout and Efficiency

One of the beauties of this implementation is how compact the state is. Each philox_engine instance requires:

counter_: 4 × 4 bytes = 16 bytes

output_: 4 × 4 bytes = 16 bytes

key_: 2 × 4 bytes = 8 bytes

STATE: 4 bytes = 4 bytes

Total = 44 bytes

This is tiny! On a GPU, you could have millions of these generators running in parallel, each consuming only 44 bytes. In comparision, traditional RNGs can take kilobytes of state per instance.

Summary

In this article, we explored Philox, a counter-based PRNG designed for parallel computing environments. We learned:

1. Why traditional PRNGs don’t parallelize well: Sequential state dependencies create bottlenecks on parallel hardware like GPUs.

2. How Philox works: By treating random number generation as a function f(counter, key), Philox allows direct computation of any random number without computing predecessors.

3. The algorithm’s core operations: Multiplication with carefully chosen constants, high-low splitting, XOR with key material, and permutation, repeated for 10 rounds to ensure statistical quality.

4. Parallelization through counter partitioning: The 128-bit counter space is split into subsequences (upper 64 bits) and offsets (lower 64 bits), allowing up to 2^64 parallel threads each generating 2^64 random numbers.

5. PyTorch’s implementation: A compact 44-byte state per engine instance, efficient batching of 4 numbers at a time, and careful handling of counter arithmetic for both CPU and GPU execution.

Share

So far, we have learned the fundamentals of instructions and registers in x86 assembly. But writing real-world programs requires memory access, so we must learn how to deal with memory. If you can master this topic, you level up as a programmer.

There are two kinds of memory where we can keep our program’s data: registers and main memory. We have already learned about using registers; they are the fastest possible memory units in the hardware. But they are very limited in numbers, while real-world code needs much more memory than that.

Apart from that, registers can only hold primitive type values. The integer registers (the 16 general-purpose ones we learned about) handle integers, while separate floating-point registers exist in x86 for floating-point operations. However, we need a way to store and access composite types, such as arrays and structs, which is only possible using main memory.

Accessing memory in assembly is a big topic, so we’ll split it into a multipart series covering each addressing mode step by step as there are several memory addressing modes, and learning to effectively use each of them is crucial for us to read and write assembly code. So, I am going to split this topic into a multipart series. In this first part, we will cover the following topics:

• Regions of memory in a process’s address space: stack, heap, and data

• Immediate addressing mode

• Direct addressing mode

In future parts, we will cover the following:

• Indirect addressing mode

• Offset-based addressing mode

• Indexed addressing mode

We’ll start by understanding how data is organized in memory before we explore addressing modes. Now, let’s dive in!

I’m also publishing this in the form an ebook (PDF). If you don’t wish to upgrade to a subscription, you can purchase the PDF using the following link. If you are a paid subscriber, you can get it at a discount (monthly subs: 20% and annual subs: 50%). Please email me for the discounted link.

Purchase PDF

Regions of Memory in Process Address Space

When programming in high-level languages, you would have learned about the concept of scope or the lifetime of a variable. For example, a global variable lives for the duration of the program; local variables are automatically destroyed when the function returns. And, you can dynamically allocate memory on the heap that lives until it is freed.

When programming in assembly, we need similar scopes. However, there is no compiler to help us out, so we must do it ourselves. These scopes can be achieved by storing data in different regions in the address space of the process. So, we must start there.

There are three main regions in the process’s address space where you can decide to store your program’s data, as shown in the following diagram.

• Stack segment: The stack segment is primarily used to implement function calls and to store function local data, such as variables and arguments. We will learn to use the stack when we talk about functions in assembly.

• Data Segment: The data segment is used to store static data. For example, whenever you create global variables or constants in your programs, the compiler may put them in the data segment. The advantage of the data segment is that it is burned as part of the program binary and loaded during startup. As a result, there is no memory allocation overhead at runtime.

• Heap Segment: The heap segment is used for dynamic memory allocation at runtime. For example, when growing an array, or creating nodes for a tree or a linked list.

In this article, we will mostly use the data segment, and we’ll cover heap and stack in future articles on dynamic memory allocation and function calls.

But, before jumping to memory access modes, we should spend a few minutes to learn how to do static memory allocation in the .data section, as we will be using static memory throughout the rest of this article.

Static Memory Allocation in the .data section

The data segment in the process’s address space is populated based on the contents in the .data section of the executable binary. When we want to create static data in our program, such as global variables or constants, we can put them in the .data section of our program.

To create a static value in the .data section, we need to do three things:

• Create a label: At the time of writing assembly, we don’t know the exact memory address of the values or instructions, so we must use labels. At linking time, the linker replaces labels with the final addresses in the object code that it generates. So, creating a label for the value gives us a way to refer to its address.

• Declare the size: We need to tell the assembler the size of the value, so that it can create that much space in the .data section. If you read the article on registers, you may recall that we have the following sizes:

  • .quad: For 8-byte values

  • .long: For 4-byte values

  • .word: For 2-byte values

  • .byte: For single-byte values

  • Apart from these, we also have the .asciz macro to create a nul-terminated ASCII string.

• Declare the value: Finally, provide the value.

The following example shows how we can create an 8-byte integer value in the .data section with the label ANSWER_TO_LIFE:

This example allocates a single 64-bit value, but it is also possible to create more complex structures. For instance, we can create a struct-like object as shown in the example below:

Read more

Performance is critical for systems programmers, and accurate benchmarking is the foundation of meaningful optimization. To truly understand where your code spends time, you need precise and low-overhead measurements, especially when a piece of code may execute in just a few hundred CPU cycles.

Most developers reach for familiar high-level timers, such as Python’s time.perf_counter() or Java’s System.currentTimeMillis(). These are convenient but rely on system calls like clock_gettime which introduce hundreds of cycles of overhead. In certain situations, this overhead can be too much. And when profiling production systems, you want the overheads to be as minimal as possible.

We need a way to read time directly from the hardware, without leaving the user space. On x86 systems, that mechanism is the rdtsc instruction. It gives us near-zero-overhead access to the CPU’s internal timestamp counter, but using it correctly requires an understanding of how modern processors execute instructions.

In this article, we’ll learn how to use rdtsc to do benchmarking. Specifically we will cover the following topics in detail:

• What rdtsc does: How it reads the CPU’s internal timestamp counter and why it provides near-zero-overhead timing.

• Understanding CPU behavior: How out-of-order execution can distort timing results and why instruction ordering matters.

• Instruction stream serialization: What it means, how the CPU reorders instructions, and how serializing instructions (like cpuid) enforce strict ordering.

• Memory fences: How lfence, sfence, and mfence provide lighter-weight ordering guarantees that help isolate measurement code.

• Combining it all: Practical example of using these mechanisms together to obtain stable and reproducible timing measurements.

By the end, you’ll know not only how to use rdtsc safely and accurately but also why these extra steps are essential for meaningful microbenchmarking.

Understanding The Timestamp Counter in the CPU

In the x86 architecture, every CPU comes with a special 64-bit counter, called the timestamp counter (TSC) that gets incremented at a fixed frequency. If you can read the value of the counter before and after the execution of a block of code, you can accurately tell how many cycles that code took to execute.

When the counter overflows, it resets to 0. However, because it is a 64-bit counter, it will take an extremely long time for it to overflow. For instance, if the counter increments at 1 GHz frequency, it will take 585 years for it to overflow.

The frequency at which the timestamp counter increments is not the same as the real CPU frequency. In the past, it used to be related to the CPU frequency but as recent CPUs started to have dynamic frequency scaling, the timestamp counter was made to tick at a fixed constant frequency to get stable measurements. For example, some of the cores on my laptop have a frequency range of 800 MHz to 4800 MHz, but the TSC ticks at 2.3 GHz.

So, how do we read the TSC? The x86 instruction set provides two instructions for doing this: rdtsc and rdtscp. But to actually measure the timing of a block of code using these is not as simple as simply slapping rdtsc before and after the code block. It is more sophisticated than that. In practice, it looks like the following code snippet:

#include <x86intrin.h>

uint32_t cpuid;
_mm_lfence();
uint64_t start = __rdtsc();

for (int i = 0; i < ITERS; i++) {
  // expensive loop body
}

uint64_t end = __rdtscp(&cpuid);
_mm_lfence();
uint64_t ncycles = end - start;

In this snippet, I have used the GCC compiler intrinsics __rdtsc and __rdtscp for invoking the rdtsc and rdtscp instructions respectively. But you may ask, what is the significance of using _mm_lfence() before and after the measurement? You may also question why we used rdtsc for reading the starting value of the TSC and rdtscp for the ending measurement. To answer these questions, we have to go deeper and think about how the processor executes instructions.

Out of Order Execution and Serializing Instructions

Let’s step back a bit and talk about how the CPU executes instructions.

Modern x86 CPUs do out-of-order execution of the instruction stream to execute multiple instructions in parallel. They do this by looking at a window of instructions in the instruction stream, identifying independent instructions and executing them in parallel. As a result, an instruction that appears later in the program order may execute much earlier than its predecessors.

For example, imagine an instruction stream as shown in the below snippet. Here, we are interested in measuring the time taken to execute instructions I4 to I6, so we have inserted an rdtsc instruction after I3 and I6.

I0, I1, I2, I3, rdtsc, I4, I5, I6, rdtsc,...

Due to the out-of-order nature of the instruction execution, we cannot guarantee if the rdtsc instructions will execute exactly in the right order. It is possible that the CPU executes the first rdtsc after I1. In that case, our measurement will include the timing of I2 and I3 as well, which is not what we want.

We need a way to force the CPU to not execute rdtsc out of its order and also ensure that all the previous instructions have finished executing when it executes rdtsc. This can be achieved by forcing serialization of the instruction stream right before rdtsc, let’s understand what that means.

Serializing the Instruction Stream

There are certain instructions in the x86 architecture that force serialization of the instruction stream. Basically, the serializing instruction acts like a barrier. The CPU cannot execute it until all the instructions appearing before it in the program have finished. Also, it cannot begin executing any instruction appearing after the serializing instruction until the serializing instruction has finished.

To be precise, a serializing instruction also requires that all the flags, registers and memory modifications must finish before it executes and all the CPU buffers must be drained.

So, if we insert such a serializing instruction before rdtsc, then we can guarantee that the rdtsc instruction will not be executed by the processor out of its actual order.

There are a few such serializing instructions available in the x86 architecture, such as:

• serialize: Serializes the instruction stream

• cpuid: used to identify the CPU model and features

• iret: returns control from an interrupt handler back to the interrupted application

• rsm: resume from system management mode

Out of these, iret and rsm are control flow modifying instructions, so you cannot use them solely for the purpose of serializing the instruction stream. In the past, cpuid was the recommended instruction for use in combination with rdtsc, and it is still an option today. However, it adds a slight overhead because the CPU needs some work to do to execute it apart from serializing the instruction stream. A much lightweight alternative is the lfence instruction that we saw in the snippet above. lfence is not a proper serializing instruction, but a memory ordering instruction. However, it serves the purpose. Let’s understand what it does.

We didn’t consider the serialize instruction because it is only available on Intel processors and missing on AMD. The instruction is purely there for serializing the instruction stream, so it is a good option. Alas, it is not portable.

The lfence instruction

An alternative to using serializing instructions with rdtsc is using memory ordering instructions, such as lfence, sfence, or mfence. These instructions add lesser overhead than pure serializing instructions, such as cpuid. Let’s understand how.

Read more

The Pi release of Python (so named because it is version 3.14, matching the digits of π) is finally here. You can go through the list of new features and major changes yourself release notes. In this post, I want to go through my top 5 favorite features of this release that I find exciting as a Python programmer and also as an engineer who loves studying system internals.

The Free Threading Python

In practical terms, the free‑threaded build allows Python programs to take advantage of multiple CPU cores concurrently, enabling true parallel execution of threads for compute‑intensive workloads.

Until Python 3.13, it was not possible to run multiple threads in parallel in Python due to the global interpreter lock (GIL), which is a global mutex inside the Python interpreter. A thread needs to acquire this lock before it can be run on the CPU. It meant that even if you had a large multicore machine, your Python process was still only using a single core. Solutions like multiprocessing were created as a workaround this limitation.

Prior to the Python 3.13 release of Python, PEP-703 was proposed to make the GIL optional. The PEP proposed a plan to introduce changes so that it would be possible to build a version of Python without the GIL by specifying a build-flag.

These changes were accepted in the 3.14 release and as a result this release of Python comes with two versions: one with the GIL still there, while the other without the GIL. If you use uv , you can install the two versions using these commands:

uv install cpython-3.14.0 #with the GIL
uv install cpython-3.14.0t #without the GIL

Note that the free threaded build of Python breaks the ABI and all the third party packages that use the C API of CPython need to be recompiled, so not all the scientific computing packages may be immediately available for use with it.

Reference Reading

The PEP-703 which describes the work behind removing the GIL is a gread read to understand the challenges behind removing the GIL and how this work has been done.

Concurrent Interpreters

A very exciting new feature in the 3.14 release is the introduction of the concurrent.interpreters module in the standard library. It allows you to run multiple Python interpreters in parallel within the same Python process. It enables yet another kind of parallelism in Python despite the GIL.

The actual implementation details behind this are tricky to explain, I will do that in another post. But if you have read my article on CPython runtime bootstrapping, you might be able to put the pieces together. But here is the executive summary.

By default, the Python process has one main interpreter and one main thread. But now, you have the ability to create multiple interpreters on demand at runtime using the concurrent.interpreters module. These other interpreters created at runtime are also referred to as subinterpreters. Creating a subinterpreter is as easy as calling the create() function of concurrent.interpreters.

import concurrent.interpreters
interp1 = concurrent.interpreters.create()

After the above call, the Python process has two interpreters inside it. Internally, the runtime tracks these using a linked list of interpreter state objects. An interpreter state represents the internal execution state of an interpreter. By providing each interpreter its own interpreter state, the runtime isolates them at Python code execution level.

To execute code on this new interpreter, we can invoke its call() method. For example:

>>> def sum(a,b):
...     return a + b
...
>>> interp1.call(sum, 10, 2)
12

However, this isn’t parallel execution because there is only one thread running in the Python process. So, the runtime simply switches the thread from executing the code inside the main interpreter to executing code inside the subinterpreter.

To execute code on the interpreter in its own thread, we can use the call_in_thread() method. Internally, this creates a new thread that executes the code in its own context. This is a non-blocking call and we cannot get the result back. So, to communicate data between interpreters, we have to create a queue using concurrent.interpreters.create_queue() method. Here is an example that puts all of this together.

>>> def add(q, a, b):
...   q.put(a+b)
...
... interp1 = concurrent.interpreters.create()
... queue = concurrent.interpreters.create_queue()
... t = interp1.call_in_thread(add, queue, 10, 2)
... result = queue.get()
... print(result)
...
12

Here, we have created a queue, and passed it to the add function. The add function puts the result in the queue. In the main interpreter, we poll the queue for the result using its get() method, which blocks until there is some data in the queue.

If you are curious about how all of this works under the hood, let me know and we can cover the internals in a future post.

Reference Reading

If you want to learn more about the runtime data structures behind this, I recommend the following article:

Remote Debugging Support

Beyond concurrency, Python 3.14 also introduces major improvements in tooling.

Debugging running Python processes has always been a pain. In order to debug it using a debugger, such as pdb, you need to manually add breakpoints in the code, then restart the process and wait for them to be hit again. In production systems, this can be infeasible.

The motivation for the new feature is to simplify this experience: with Python 3.14, you can attach to a running process using python -m pdb -p <pid>, eliminating the need to restart it.

Technically, the CPython interpreter already had provisions to allow remote processes to connect to it and navigate its runtime state. This is how remote profilers, such as scalene, pyspy and others work. As part of PEP-768, this framework has been extended to allow debuggers to connect and debug the Python interpreter.

A debugger can now attach to a Python process and update specific fields in its runtime data structures to signal that it wants to begin debugging. When the interpreter detects this, it provides a debug prompt where you can set breakpoints and debug as usual.

While pdb has already been updated to support remote debugging, this framework also exposes an API, sys.remote_exec, so external debuggers can leverage this functionality without needing low-level C integration.

Reference Video

In a past live session, I talked about how remote profilers work which is exactly how remote debugger implementation has also been done. So, if you are curious, give it a watch.

Incremental Garbage Collection

Complementing the concurrency and debugging improvements discussed earlier, this feature enhances runtime stability and responsiveness by addressing garbage collection performance.

In a past article, I explained in detail the cost of a full heap scan by the garbage collector in CPython. Needless to say it is expensive, and moreover, it also introduces unpredictable latency delays in the performance of your APIs, because when the GC is running, the interpreter does not execute any Python code. Incremental garbage collection makes the GC overhead predictable, resulting in smoother performance for latency-sensitive workloads.

Let’s first understand how the GC used to work before this change. There were three collectable generations: young generation, old generation, and the oldest generation. There were configurable thresholds for each generation that would define when the GC would scan each of those generations. For example, the young generation would be scanned once the number of objects in it exceeds 10,000.

Any object that survives a scan of the young generation gets promoted to the first old generation. The first old generation gets scanned when the young generation has been scanned a configured number of times, such as 10 times. When that happens, the GC scans both the young gen and the first old gen. Any object that survives a scan of the first old generation gets promoted to the 2nd old generation (also known as the oldest generation).

The oldest generation is scanned when the first old generation has been scanned a configured number of times. When that threshold is reached, the GC performs a full heap scan, i.e. all the three generations. Naturally, this gets expensive.

Incremental garbage collection improves this. It reduces the number of GC generations to just two: young and old. On each GC cycle, the collector scans the young generation and a fraction of the old generation. This way, the amount of work that the GC does on each cycle becomes consistent and it eliminates those long pauses and latency spikes that were there due to a full heap scan.

Reference Reading

If you want to read more about CPython’s garbage collector, I recommend the following articles:

Tail Calling Interpreter

Finally, my favorite change as part of this release is the tail calling interpreter. It is a rewrite of the bytecode dispatch loop in the CPython virtual machine and improves performance of Python code execution by ~5%.

The bytecode dispatch loop is the heart of the interpreter where the bytecode instructions of your compiled Python program are evaluated. The faster this loop runs, the faster your Python program executes, so performance improvement in this are are always very exciting to understand. I have already written a very detailed article on the design and implementation of the dispatch loop in CPython, and I have another article in progress to explain the tail calling interpreter. So, I will be brief here.

Your Python program gets compiled to a sequence of bytecode instructions. For example, the following snippet shows the bytecode instructions for a single line of code: a + b. So, the bytecode dispatch loop iterates over these instructions one by one and executes them.

>>> import dis
>>> dis.dis(”a + b”)
  0           0 RESUME                   0

  1           2 LOAD_NAME                0 (a)
              4 LOAD_NAME                1 (b)
              6 BINARY_OP                0 (+)
             10 RETURN_VALUE

The most obvious way of writing this loop is using a switch case. The problem with that is that Python has hundreds of bytecode instructions, so this switch case is huge. Optimizing such large functions is hard for compilers. For example, it cannot allocate registers optimally and some of the key variables can get spilled onto the stack, resulting in poor performance.

CPython also has a computed goto based implementation of the dispatch loop but that also suffers from the same problem. If you are not familiar with computed goto based dispatch loop, read my article on the design and implementation of the CPython dispatch loop.

The tail calling interpreter solves this by separating the implementation of each bytecode instruction into an individual function. For example, there is one function for handling LOAD_NAME, another for BINARY_OP, and so on.

This implementation is called tail calling interpreter because of the way these functions are written. At their end, instead of returning, these functions call the function for the next bytecode instruction. They do this by looking up a function pointer table using the next bytecode instruction as an index. The signature and return value of each of these functions is identical, and because these calls occur at the end of the function, they are tail calls. The compiler can optimize these tail calls and convert them into jumps, which avoids the overhead of function calls.

This implementation improves performance due to one fundamental reasons:

Overall, this has shown improvement over the previous switch case and computed goto based implementations. However, it requires compiler support for performing tail call optimization which is not present in all compilers. As a result, right now the feature is opt-in only and you need to build CPython from source using a supported compiler, such as clang 19.

Reference Reading

If you want to understand the internals of the CPython bytecode interpreter and the dispatch loop, read the following article:

Wrapping Up

Although there are many other new features and improvements in this release of Python, I picked these because of my interest in Python internals and performance. Apart from that, changes such as remote debugger and GIL removal are also very exciting to understand from an engineering point of view. Studying these can give you insights that can help you improve as an engineer.

I have plans to write about some of these in future posts. But if you would like me to cover something specific, let me know.

Share

When working with Python (and many other languages), you often rely on the runtime to manage memory for you. Most of the time this works invisibly, but certain patterns such as objects that reference each other in cycles, long lived caches, or subscriber lists can create memory leaks if not handled carefully.

This happens because Python always creates strong references to objects, which means the object will be kept alive as long as all such strong references exist in the program. But when used in cyclic data structure, or in caches, these strong references can unnecessarily delay the deallocation of these objects.

Weak references provide a way to refer to objects without preventing them from being garbage collected. They let you build caches that automatically empty, subscriber lists that clean themselves up, and other data structures that will not accidentally extend object lifetimes.

In this article we will explore what weak references are, why they matter, and how to use them in Python. We will start with a review of reference counting, look at its limitations, and then dive into weak references and their practical uses.

CodeRabbit: Free AI Code Reviews in CLI (Sponsored)

As developers increasingly turn to CLI coding agents like Claude Code for rapid development, a critical gap emerges: who reviews the AI-generated code? CodeRabbit CLI fills this void by delivering senior-level code reviews directly in your terminal, creating a seamless workflow where code generation flows directly into automated validation. Review uncommitted changes, catch AI hallucinations, and get one-click fixes - all without leaving your command line. It’s the quality gate that makes autonomous coding truly possible, ensuring every line of AI-generated code meets production standards before it ships.

Get Started Today

A Review of Reference Counting

Many languages either use reference counting as a mechanism to manage runtime memory or they provide first class primitives to do use reference counting.

In this scheme, every object has an associated reference count which means the number of places it is being used. For example, when you create an object and assign it to a variable it will have a reference count of 1. When you assign it to another variable or pass it to another function, its reference count will go up by 1.

Similarly, when a variable goes out of scope, or a function call returns then its reference count gets decremented. If the reference count of the object reaches 0, it gets deallocated or garbage collected.

CPython uses reference counting for managing the memory of its runtime. But other languages also offer it as well. For example, in C++ or rust when you use a smart pointer, it uses reference counting under the hood, the compiler generates code that increments and decrements the reference count of the objects.

If you want to understand how CPython implements reference counting internally, you can check out my article on that topic:

Limitations of Reference Counting

Reference counting works well for most cases, but it is not a complete solution. Its simplicity comes with trade‑offs, and understanding these limitations helps motivate why Python also offers weak references.

One of those limitations is cyclic references. Cyclic references exist when objects hold references to each other in a cycle, e.g. in a graph data structure. But you can also end up creating cyclic references accidentally in complex systems. In such cases, the objects that are part of the cycle will never get freed until the cycle is broken. This is why CPython also implements a cycle breaking garbage collector (GC) that runs periodically, scans the objects for cycles and if it detects cycles that are no longer referenced from anywhere else, then it breaks them so that those objects can be freed.

Cyclic references can be problematic for performance because memory usage remains high until the GC runs, and the GC scan itself can be expensive (depending on the number of objects it needs to scan).

We can understand this with the help of an example. Consider the following code

Let’s break it down:

• The MyNode class implements a linked list node with a next field.

• print_node_objects is a utility function. It finds all the MyNode objects that are currently alive and then prints their referrers, i.e., who is holding a reference to them.

  • It uses gc.get_objects() to get the list of all the currently alive objects in the Python interpreter and filters it down by checking for their type and selecting only MyNode type objects.

  • It finds the referrers to an object by using the gc.get_referrers() method which returns a list of referrer objects. We are filtering this list by type because during the call, the gc module itself becomes a referrer and we want to filter it away.

• In the main function we call the test1() function that creates two MyNode objects, prints their reference counts and returns. After returning from test1, we call print_node_objects() to see if there are any MyNode type objects that are still alive.

If you run this program, you should see an output like the following:

➜ uv run --python 3.13 --  cycles.py
n1 refcount: 2
n2 refcount: 2
n1 is being deleted
n2 is being deleted
No MyNode objects found

This is pretty much the expected output, but let’s spend a moment to ensure we don’t miss anything.

• We see that the reference count for both n1 and n2 is 2. You might expect it to be 1 but it is 2 because during the call to sys.getrefcount, the object’s reference count gets incremented.

• We see that the __del__ method of both the object gets called and prints a message. This happens because n1 and n2 are local variables inside test1(), and when it returns, its stack frame gets destroyed which results in the reference counts of all of its local objects (parameters and locally created variables) being decremented. In this case, because n1 and n2 reached reference count 0, they were deallocated and their __del__ method was called.

• Finally, in main(), when print_node_objects() is called, we see that it does not find any MyNode objects on the heap that are still alive.

Next, we can do another test that creates a cycle between n1 and n2 and see that the objects stay alive after the return from the test function. The following figure shows the updated code where I’ve added a new function test2() and then calling it from main.

If we run this program, we should see the following output:

➜ uv run --python 3.13 --  cycles.py
n1 refcount: 2
n2 refcount: 2
n1 is being deleted
n2 is being deleted
No MyNode objects found
---------------------
n1 refcount: 3
n2 refcount: 3
n1 exists with referrers: [’n2’]
n2 exists with referrers: [’n1’]
n1 is being deleted
n2 is being deleted

Let’s focus on the output after the call to test2().

• We see that in test2(), the reference count for n1 and n2 is 3, one higher than what it was in test1(). This is due to n1.next creating a reference to n2 and n2.next creating a reference to n1.

• We also see that when test2() returns, the __del__ method of n1 and n2 is not called, it means that those objects are not deallocated and are still alive. This happened because during the return, the interpreter would decrement their reference count but this time the reference count does not reach 0.

• After return from test2(), when we call print_node_objects(), we see that it tells us that the MyNode objects we created for n1 and n2 are still alive. We can also see that they are alive because they are holding cyclic reference to each other.

• n1 and n2 finally get destroyed as the program ends because the CPython interpreter runs the GC before shutting down.

To avoid such cyclic references from leaking memory, CPython includes a garbage collector that periodically runs, detects cycles that are no longer from anywhere else, and breaks them so that the objects that are part of the cycle can get deallocated. You can verify it yourself by inserting a gc.collect() call after the call to test2() in the above program.

If you want to understand how the CPython garbage collector detects and breaks cycles, read my article on its internals:

However, there are other ways to avoid such pitfalls of reference counting and weak references is one of them. Let’s understand what they are and how they work.

Understanding Weak References

Weak references are on the opposite spectrum of strong references. A weak reference does not increase the reference count of the underlying object, so it enables you to use an object without prolonging the lifetime of the object.

When the object’s reference count goes to 0, it can get deallocated even if there are weak references to it that are still being used. Naturally, this requires that when using a weak reference to an object, we always need to check if the underlying object is still alive.

In Python, to create weak references, we need to use the weakref.ref() function from the weakref module and pass the object for which we want to create a weak reference. For example:

n1_weakref = weakref.ref(n1)

weakref.ref() creates a weak reference to the given object and returns us a callable. To access the underlying object we need to invoke this callable everytime. If the object is still alive, it returns a handle to the object, otherwise it returns None. For example:

if n1_weakref():
  print(f"name: {n1_weakref().name}")
else:
  print("n1 no longer exists")

The following figure shows a full example of creating a weak reference and accessing it in our running linked list example.

Output:

➜ uv run --python 3.13 --  weakref_cycles.py
n1 refcount: 2
n1 refcount: 2
n1’s name: n1
n1 is being deleted
n1 no longer exists
---------------------
No MyNode objects found

From the output we can confirm a few things:

• Creating a weak reference does not increase the object’s reference count

• A weak reference does not prevent the object from being deallocated if its reference count goes to 0 (in the example we deleted n1 and after that we were not able to access it using the weak reference.).

I leave the problem of fixing the cyclic reference that we created in test2() as an exercise for you.

Other Use Cases of Weak References

So far we’ve seen weak references as a tool for avoiding cycles, but their utility goes well beyond that. The weakref module also provides ready-made containers built on top of weak references. These containers, WeakValueDictionary and WeakSet, help you manage auxiliary data structures that should not extend the lifetimes of their contents. They solve practical problems such as caching, registries, and subscriber lists, where automatic cleanup is not just convenient but essential for avoiding leaks.

WeakValueDictionary

The weakref module provides WeakValueDictionary, which looks and behaves like a normal dictionary but with an important twist: its values are held only through weak references. If a value is no longer strongly referenced anywhere else, the dictionary entry disappears automatically.

This makes WeakValueDictionary a natural fit for caching and memoization. Imagine you compute expensive results or load large data structures and want to reuse them if they are still in memory. At the same time, you don’t want the cache itself to keep them alive forever. A WeakValueDictionary strikes that balance: it holds onto results only as long as the rest of the program does.

Another classic application is object interning or registries. For example, you may want to ensure there is only one canonical object representing a resource (like a symbol table entry, database connection, or parsed schema). By using a WeakValueDictionary, you avoid artificially extending the lifetimes of those resources.

Here’s a simple illustration:

import weakref

class Data:
    def __init__(self, name):
        self.name = name
    def __repr__(self):
        return f"Data({self.name})"

cache = weakref.WeakValueDictionary()
obj = Data("expensive_result")
cache["key"] = obj

print("Before deletion:", dict(cache))

# Drop the strong reference
obj = None

print("After deletion:", dict(cache))

Output:

Before deletion: {'key': Data(expensive_result)}
After deletion: {}

Notice how the cache entry vanishes automatically once the last strong reference goes away. There is no need for manual cleanup. Under the hood, this is implemented with weakref callbacks—the same mechanism we’ll see in the callback section.

WeakSet

Another container provided by the weakref module is WeakSet. This is similar to a regular set, except that it holds weak references to its elements. If an object is garbage collected, it will automatically vanish from the set.

One scenario where this is very handy is when you want to keep track of subscribers, observers, or listeners. These are objects that register interest in events produced by another object (often called the publisher). For instance:

• GUI frameworks: widgets listen to events such as theme changes or window resizes.

• Event buses: services subscribe to log events, metrics, or domain events.

• Plugin systems: plugins register callbacks at load time to respond to hooks.

• Background services: transient sessions (e.g., WebSocket connections) listen for updates from a long‑lived manager.

In all these cases, subscribers are often short‑lived, while the publisher lives much longer. Using a regular set to hold them risks memory leaks, because a strong reference in the set will keep the subscriber alive even when the rest of the program has forgotten it. With a WeakSet, the garbage collector automatically removes subscribers that are no longer strongly referenced anywhere else, so you don’t need explicit unsubscribe logic in every shutdown path.

Here’s a simple example:

import weakref

class Listener:
    def __init__(self, name):
        self.name = name
    def __repr__(self):
        return f"Listener({self.name})"

listeners = weakref.WeakSet()

l1 = Listener("A")
l2 = Listener("B")
listeners.add(l1)
listeners.add(l2)

print("Before deletion:", list(listeners))

# Remove one listener
l1 = None
import gc; gc.collect()

print("After deletion:", list(listeners))

Output:

Before deletion: [Listener(A), Listener(B)]
After deletion: [Listener(B)]

This pattern is often extended into a publisher–subscriber model:

class Publisher:
    def __init__(self):
        self._subs = weakref.WeakSet()
    def subscribe(self, sub):
        self._subs.add(sub)
    def notify(self, payload):
        for s in list(self._subs):
            s.handle(payload)

class Subscriber:
    def __init__(self, name):
        self.name = name
    def handle(self, payload):
        print(self.name, "got:", payload)

pub = Publisher()
sub = Subscriber("one")
pub.subscribe(sub)

pub.notify({"event": 1})  # delivered
sub = None                  # drop last strong ref
import gc; gc.collect()

pub.notify({"event": 2})  # nothing printed; WeakSet cleaned itself

Using WeakSet here avoids leaks and simplifies lifecycle management. A caveat is that only weak‑referenceable objects (i.e., user‑defined classes) can be added; built‑ins like int or tuple won’t work. If your class uses __slots__, include __weakref__ to allow weak references.

Callbacks on Weak References

Another useful feature of weakref.ref is the ability to attach a callback. A callback is a function that gets invoked automatically when the referent object is about to be finalized. This can be handy if you want to clean up auxiliary data structures or release resources when an object goes away.

import weakref

class Resource:
    def __init__(self, name):
        self.name = name
    def __repr__(self):
        return f"Resource({self.name})"

def on_finalize(wr):
    print("Resource has been garbage collected:", wr)

obj = Resource("temp")
wr = weakref.ref(obj, on_finalize)

print("Created weak reference:", wr)

# Drop strong reference
obj = None

# Force GC for demo purposes
import gc; gc.collect()

Output:

Created weak reference: <weakref at 0x75f6773870b0; to ‘Resource’ at 0x75f677c4ee40>
Resource has been garbage collected: <weakref at 0x75f6773870b0; dead>

Here, the on_finalize callback is called once the Resource instance is about to be collected. The weak reference itself becomes dead afterwards. This pattern is useful when you want to implement custom cleanup logic tied to an object’s lifecycle.

It’s also worth noting that containers like WeakValueDictionary and WeakSet use this same mechanism internally: they attach callbacks to their weak references so that entries are automatically removed when the referent objects are finalized.

Conclusion

Weak references are not a tool you’ll reach for every day, but when you need them they solve very real problems. At the lowest level, weakref.ref lets you point to an object without affecting its lifetime, and you can even attach a callback to run cleanup code at the moment it is collected. Building on that primitive, Python’s WeakValueDictionary and WeakSet give you higher level containers for caches, registries, and subscriber lists that automatically clean themselves up when their contents go away.

To summarize the differences:

Together, these features make it possible to build memory‑friendly systems that avoid leaks, reduce bookkeeping, and respect the natural lifetimes of your objects. Understanding weak references and knowing when to apply them will help you write code that is both safer and more efficient.

Further Reading

• Python documentation on weakref

• Python garbage collector documentation

• “Fluent Python” by Luciano Ramalho – includes in depth coverage of weak references and how to use them

Share

A recurring theme of this newsletter is going under the hood: how interpreters, compilers, and runtimes actually work, and what performance trade‑offs they force on us. Python is a perfect case study: it’s beloved for its simplicity, but that same simplicity often means poor performance when the workloads get serious.

That’s why I’m really excited to share this guest post by , founder of Muna. In this piece, he looks at how Python could be pushed beyond its usual limits of speed and portability, laying out a compiler that turns ordinary code into fast, portable executables.

Instead of building another JIT or rewriting everything in C++, his approach generates optimized kernels while keeping the Python source unchanged. This ties directly to themes I’ve written about before, such as CPython internals, and performance engineering. All of those pieces showed why understanding systems at the lowest level matters. Yusuf’s work demonstrates the payoff of that mindset: the ability to design and build new systems on top of that knowledge.

Introduction

I first met Abhinav at the start of 2024, trying to learn more about how the Python interpreter worked under the hood. I had reached out to Abhinav with a singular goal in mind: to build something that could compile pristine Python code into cross-platform machine code.

This idea has been attempted in many forms before: runtimes (Jython, RustPython), DSLs (Numba, PyTorch), and even entirely new programming languages (Mojo). But for reasons we will explore later in this article, we needed something that could:

1. Compile Python entirely ahead-of-time, with no modifications.

2. Run without a Python interpreter, or anything other interpreter.

3. Run with minimal overhead compared to a raw C or C++ program.

4. Most importantly, run anywhere—server, desktop, mobile, and web.

In this article, I will walk through how this seemingly crazy idea came about, how we began building a solution, how AI happened to be the missing piece, and how we’ve grown to serve thousands of unique devices each month with these compiled Python functions.

Containers Are the Wrong Way to Distribute AI

I got my start in AI research around 2018, back when we called it “deep learning”. I had taken a year off from college and was coming off my first startup experience as co-founder of a venture-backed proptech startup that would later get acquired. One very interesting problem I had encountered in this journey was image editing for residential listings. Each month, a real estate photographer would outsource thousands of photos of homes to be hand-edited in Photoshop and Lightroom, before being posted on the regional MLS or on Zillow.

I teamed up with an old friend and we set out to build a fully automated image editor, using a new class of vision AI models called Generative Adversarial Networks (GANs). We would train our custom model architectures on our datasets, then test rigorously to ensure that the models worked correctly. But when it came time to get these AI models into the hands of our design partners, we simply got stuck. I spent the majority of my time trying to get our models into something we could distribute very easily. But after months of wrangling with Dockerfiles and third-party services, it became crystal clear to me: containers are the wrong unit of distribution for AI workloads.

To understand why, we need to look into the container. Containers are simply self-contained Linux filesystems with runtime isolation and resource management. So when deploying our AI model as a container, we would package up the inference code, the model weights, all the Python package dependencies, the Python interpreter itself, and other required software into what was effectively a snapshot of a full Linux operating system.

But what if instead of making a self-contained operating system, we made a self-contained executable that ran only our AI model and nothing else? The benefits here would be significant: We could ship much smaller containers that started up much faster, because we wouldn’t have to include unnecessary Python packages, the Python interpreter itself, or any of the other unnecessary cruft that gets bundled into the container. But even more importantly, not only could we run these executables on our Linux servers—we could run them anywhere.

Arm64, Apple, and Unity: How It All Began

I started programming at the age of eleven, thanks to my dad who vehemently refused to buy me a PlayStation 2 out of fear that my grades would drop. Out of an extreme stubbornness, inherited from him and my mom, I had decided that if he was not going to buy me a game console, then I would simply build the games myself1. I was lucky enough to find a game engine that was intuitive, allowed developers to build once and deploy everywhere, and most importantly, was free to use: Unity Engine.

In late 2013 Apple debuted the iPhone 5S, its first device featuring the relatively new armv8-a instruction set architecture. Unlike prior devices, this was a 64-bit architecture running on ARM. With it, apps could address much more memory, and benefit from a myriad of performance gains. As such, Apple quickly mandated that all new apps be compiled for arm64.

Unity, with its massive developer ecosystem, was thrown into a tailspin. To understand why, we need some context on how Unity works: Because Unity is a game engine, objects within the game can be scripted to have custom behaviors. C# was Unity’s chosen scripting language for these behaviors. But C# does not compile to object code, so it needs a virtual machine to execute at runtime (sound familiar?). Unity used Mono for this purpose, but Mono did not support arm64.

Unity embarked on a journey to build what I still consider to be its greatest engineering feat: IL2CPP. As its name implies, the IL2CPP compiler would take in Common Intermediate Language bytecode (i.e. the intermediate representation generated by the C# compiler); then emit equivalent C++ source code. Once you had C++ source code, you could compile that code to run just about anywhere: from Nvidia GPUs and WebAssembly; to Apple Silicon and everything in-between.

We set out to build the exact same, for Python.

Sketching Out a Python Compiler

At a high-level, the compiler would:

1. Ingest plain Python code, with no modifications.

2. Trace it to generate an intermediate representation (IR) graph.

3. Lower the IR to C++ source code.

4. Compile the C++ source code to run across different platforms and architectures.

Before jumping in, you might be wondering: why bother generating C++ first? Why not just go from IR to object code?

Going back to why we started on this journey, our main focus with Muna has been on compute-intensive applications, especially AI inference. If you’ve spent time in this space, you’re familiar with technologies like CUDA, MLX, TensorRT, and so on. But there are so many more frameworks, libraries, and even undocumented ISAs that applications can leverage to accelerate everything from matrix multiplication to computer vision.

We wanted to design a system that would allow us leverage as many ways to perform some computation as we might have available on given hardware. We’ll show you how we achieved this, and how this design gives us a novel, data-driven approach to performance optimization.

Building a Symbolic Tracer for Python

The first step in building our compiler is to build a symbolic tracer. The tracer’s job is to take in a Python function and emit an intermediate representation (IR) graph that fully captures control flow through the function.

Our very first prototypes were built upon the PyTorch FX symbolic tracer, introduced in PyTorch 2.0. Their symbolic tracer was built off PEP 523, a feature in CPython that allowed developers in C to override how bytecode frames are evaluated by the interpreter. I won’t go into too much detail here, as it is a marvel of engineering in its own right, but in summary PEP 523 enabled the PyTorch team to register a hook that could record every single function call as it was being evaluated by the interpreter:

Confessions of a Code Addict

The Design & Implementation of the CPython Virtual Machine

For every bytecode compiled language, the most interesting part of its implementation is its virtual machine (also referred to as the bytecode interpreter) where the bytecode execution takes place. Because this is such a crucial part of the language machinery, its implementation has to be highly performant. Even if you are not a compiler engineer, learning about such internal implementation can give you new performance tricks and insights that you may be able to use in other places of your job. And, if you are a compiler engineer then you should always look around how other languages are implemented to pickup implementation details that you may not be aware of…

Read more

2 years ago · 46 likes · Abhinav Upadhyay

Unfortunately, TorchFX had two significant drawbacks that required us to build a custom tracer. The first is that once you hook into the CPython interpreter to record your PyTorch function, you have to actually run said function. For PyTorch, this was not an issue because you could invoke your function with so-called “fake tensors” that had the right data types, shapes, and devices, but allocated no memory. Furthermore, this way of running a function in order to trace it would be perfectly inline with how their legacy serialization APIs worked (torch.jit and torch.onnx).

Since we needed the ability to compile arbitrary Python functions, of which only a tiny subset (or none) could be PyTorch, we would need a similar mechanism for having developers provide us with their inputs to use for tracing. But unlike PyTorch, we could not create a fake image, or fake string, or fake whatever. To us, this became a dead end.

The second challenge was that even when we created fake data as inputs to the TorchFX tracer, we realized that it could only record PyTorch operations. We would have to heavily modify and extend the tracer to support tracing through arbitrary functions across hundreds and thousands of Python libraries. As such, we settled on building a tracer that would instead capture a Python function by parsing its abstract syntax tree (AST). Take an example function:

Our tracer would first extract an AST like so:

It would then step through, resolve all function calls (i.e. figure out what source library each function call belongs to), then emit a proprietary IR format. Currently, our symbolic tracer supports static analysis (via AST parsing); partial evaluation of the original Python code; live value introspection (using a sandbox), and much more. But somehow, it’s the least interesting part of our compiler pipeline.

Lowering to C++ via Type Propagation

This is where things get really interesting. Python is a dynamic language, so variables can be of any type, and those types can change easily:

C++ on the other hand, is a strongly-typed language, where variables have distinct, immutable types that must be known when the variable is declared. While bridging both of these languages might seem like an intractable problem, there’s actually a key insight we can take advantage of:

When we invoke a Python function with some given inputs, we can uniquely determine the types of all intermediate variables within that function2:

If we know that the inputs x and y are float instances, then we know that the resulting type of their multiplication (i.e., tmp_1) is uniquely determined by whatever the operator.mul function returns. But how do we define operator.mul and get its return type? That’s where C++3 comes in.

From the above, we now know that tmp_1 must be a float. We can repeat this process for the addition call (tmp_1 + z) to get the final result.

At this point, it’s worth taking a moment to reflect on what we have created thus far:

1. We can take a Python function and generate an intermediate representation (IR) that fully captures what it does.

2. We can then use parameter type information; and a C++ implementation of a Python operator (e.g. operator.mul); to fully determine the type of the first intermediate variable in our Python function.

3. We can repeat (2) for all subsequent intermediate variables in our Python function, until we have propagated types throughout the entire function.

Seeding the Type Propagation Process

One point worth expanding upon is how we get the initial parameter type information to kickstart the type propagation process. In the example above, how do we know that each of x, y, and z are float instances?

After prototyping with a few different approaches, we settled on PEP 484 which added support for type annotations in the Python language. Python itself completely ignores these type annotations, as they are not used at runtime4. And while they solved the problem of seeding type propagation, they came with two major drawbacks: first, they conflict with our most important design goal, because using them requires developers to modify their Python code a little5:

The code doesn’t look too different, and some argue that using type annotations makes for writing better Python code (we mandate them at Muna). The second problem was that in order to design a simple and modular interface for consuming the compiled functions, we would have to constrain the number of distinct input types that could be used by developers6. Ultimately, we decided this was a reasonable compromise with nice ergonomics.

Building a Library of C++ Operators

At this point, you might have realized a glaring issue in our design: we need to write C++ implementations for potentially tens or hundreds of thousands of Python functions across different libraries. Thankfully, this is a lot less complicated than you might think. Consider the function below:

When compiling cosecant, we see that there are function calls to sin and reciprocal. Our compiler first checks if it can trace through each function call. In the case of sin, we don’t have a function definition for it (only an import), so we cannot trace through it7. This forms a leaf node that we must implement manually in C++. We can trace through the call to reciprocal, so we do and get an IR graph for it. This can then be lowered and used at other call sites.

The key insight above is that most Python functions our compiler will encounter are composed of a smaller set of elementary functions. What accounts for the large variety of code in the wild is not the unique number of elementary functions that make them up; rather, it’s the different arrangements of these elementary functions.

Still, you could argue that there are potentially thousands of these elementary functions across different libraries that we would have to cover, and you would be 100% correct. Thankfully, we now have an amazing tool that makes this an easy problem to solve: AI-powered code generation.

Today’s LLMs are capable of writing verifiably-correct, high-performance code across a wide variety of programming languages. As such, we’ve been building infrastructure to constrain the code they generate, test the code to ensure correctness, and handle ancillary logic for things like dependency management and conditional compilation. So far, we have used AI to generate implementations of hundreds of Python functions across popular libraries like Numpy, OpenCV, and PyTorch.

Performance Optimization via Exhaustive Search

The final topic worth discussing is performance optimization. Most popular approaches here involve rolling out hand-written code (e.g. Assembly or PTX); using heterogenous accelerators (e.g. GPU, NPU); doing heuristic-based algorithm selection at runtime (e.g. convolution algo search in ArmCL and cuDNN); or some combination thereof.

From our past experience building extremely low-latency computer vision pipelines for embedded systems, we have learned a very bitter lesson8: effective performance optimization is always empirical. The latency of a given operation on some given hardware depends on so many factors that the only way to know for sure it to simply test every single approach you have. The only reason why engineering teams don’t do this is because it is impractical: you would have to rewrite your code tens or hundreds of times then test each variant…but wait!

Earlier, we went over how we propagated types through a Python function with the help of a C++ operator. What I didn’t mention was that we don’t just use one C++ operator; we use as many as we can write (*ahem* generate). So instead of this:

What really happens is this:

Each path from start to result is a unique program, guaranteed to be correct with respect to the original Python function. But each C++ operator (colored rectangles) could be powered by different algorithms, libraries, and even hardware accelerators. Let’s walk through a concrete example:

The function above resizes an input image to 64x64 with bilinear resampling using the torchvision library. When compiling this function for Apple Silicon (macOS, iOS, or visionOS), we have a range of approaches and libraries to choose from, including:

The above is just a small selection, as the possibilities for implementing a bilinear resize operation on Apple Silicon are numerous (e.g. using the GPU, Neural Engine). The key here is that we can generate as many of these as possible (thanks to LLM-powered codegen), then emit compiled programs that use each one—with absolutely no limits. So in the example above, the user’s Python function would be emitted as four unique programs for Apple Silicon alone. In our real-world testing, we have seen a single Python function be emitted as almost 200 unique programs across 9 compile targets.

From here, we can easily test each compiled function to discover which one runs the fastest on given hardware. We gather fine-grained telemetry data, containing latency information for each operation, and use this data to build statistical models to predict which variant runs the fastest. There are two significant benefits in this design:

1. We can optimize code purely empirically. We don’t make any assumptions about which code might perform best; and we don’t need a separate performance tuning step after generating code. We simply ship out every compiled binary we have, gather telemetry data, and use this to discover which one is the fastest.

2. We benefit from network effects. Because the C++ operators are shared among thousands of compiled functions; and because we ship these compiled functions to hundreds of thousands of unique devices across all of our users; we have tons of data that we can use to optimize every piece of code we generate.

For our users, this will feel like their compiled Python functions running faster over time, entirely on autopilot.

Designing a User Interface for the Compiler

Now, we have to wrap up everything we’ve covered above into a user interface. Our most important guiding principle was to design something with near-zero cognitive load. Specifically, we didn’t want developers to have to learn anything new to use the compiler. We decided to go with PEP 318, decorators:

Developers could simply decorate their Python function with @compile to specify the compilation entrypoint. Then, they would compile the function9 and all its dependencies using the CLI:

We fell in love with the decorator paradigm from seeing how developers strongly preferred expressing complex infrastructure as code10. Furthermore, it was a familiar form factor within the Python ecosystem, evidenced by its use within Numba and PyTorch. With the decorator, our CLI could find the compilation entrypoint function, and use that as a springboard to crawl through all other dependency code (both first-party as provided by the developer, and third-party packages installed via pip or uv).

The @compile decorator would also serve as the primary customization point for developers compiling their function. Beyond the required tag (which uniquely identifies the function on our platform) and description, developers could provide a sandbox description to recreate their local development environment (e.g. installing Python packages, uploading files); along with metadata to assist the compiler during codegen (e.g. running PyTorch AI inference with ONNXRuntime, TensorRT, CoreML, IREE, QNN, and more).

Once compiled, anyone can run the compiled function anywhere11:

Closing Thoughts

In all candor, we still have a high level of disbelief that any of this actually works. That said, the compiler has a lot of standard Python features that are partial or missing: exceptions, lambda expressions, recursive functions; and classes. The through-line connecting these missing features is our type propagation system. While type propagation works for simple functions with unitary parameter and return types, it requires additional consideration for composite types (e.g. unions) and higher-order types (e.g. classes, lambda expressions).

The other significant item we are still figuring out is the debugging experience. The good news for us is that we guarantee that developers’ Python code will work as expected once compiled, absolving them of any responsibility to debug the code at runtime. This is similar to how developers who use Docker or other containerization technologies simply expect everything to work—almost nobody debugs their Docker image layers. The bad news is that because we enable developers run their Python code anywhere, we have to figure out how to write extremely safe code; and how to gather fine-grained, symbolicated trace data when some function raises an exception. This is further complicated by the fact that because we have to deliver the smallest and fastest compiled binaries possible, we compile generated code with full optimizations, inevitably stripping out valuable debug information.

It has not all been difficult though, especially because the evolving C++ standard has been a major boon for us. Muna would not exist without C++20, because our code generation relies extensively on std::span, concepts, and most importantly, coroutines. And we’re dying for broad C++23 support, because we use std::generator to support streaming, <stdfloat> to support float16_t and bfloat16_t, and <stacktrace> to support Python exceptions.

On a final note, if you are currently deploying embedding models or object detection models in your organization, or if you find any of this work interesting, we would love to chat with you. We’d love for more developers to use the compiler on problems and programs we haven’t yet run ourselves; and we love to meet developers who enjoy the mundane, low-level worlds of Python, C++, and everything in-between.

Come Chat with Us

System calls are how user programs talk to the operating system. They include opening files, reading the current time, creating processes, and more. They’re unavoidable, but they’re also not cheap.

If you’ve ever looked at a flame graph, you’ll notice system calls often show up as hot spots. Engineers spend a lot of effort cutting them down, and whole features such as io_uring for batching I/O or eBPF for running code inside the kernel exist just to reduce how often programs have to cross into kernel mode.

Why are they so costly? The obvious part is the small bit of kernel code that runs for each call. The bigger cost comes from what happens around it: every transition into the kernel makes the CPU drop its optimizations, flush pipelines, and reset predictor state, then rebuild them again on return. This disruption is what makes system calls much more expensive than they appear in the source code.

In this article, we’ll look at what really happens when you make a system call on Linux x86-64. We’ll follow the kernel entry and exit path, analyse the direct overheads, and then dig into the indirect microarchitectural side-effects that explain why minimizing system calls is such an important optimization.

CodeRabbit: Free AI Code Reviews in CLI (Sponsored)

As developers increasingly turn to CLI coding agents like Claude Code for rapid development, a critical gap emerges: who reviews the AI-generated code? CodeRabbit CLI fills this void by delivering senior-level code reviews directly in your terminal, creating a seamless workflow where code generation flows directly into automated validation. Review uncommitted changes, catch AI hallucinations, and get one-click fixes - all without leaving your command line. It's the quality gate that makes autonomous coding truly possible, ensuring every line of AI-generated code meets production standards before it ships.

Get Started Today

Background on System Calls

Let’s start with a quick overview of system calls. These are routines inside the kernel that provide specific services to user space. They live in the kernel because they need privileged access to registers, instructions, or hardware devices. For example, reading a file from disk requires talking to the disk controller, and creating a new process requires allocating hardware resources. Both are privileged operations, which is why they are system calls.

Calling a system call requires a special mechanism to switch execution from user space to kernel space. On x86-64 this is done using the syscall instruction, where you place the syscall number in rax and the arguments in registers (rdi, rsi, rdx, r10, r9, r8), then invoke syscall:

# set args for calling read syscall
movq $1, %rax
movq $1, %rdi
movq $buf, %rsi
movq $size, %rdx
syscall # we enter the kernel here
movq %rax, %rbx

On encountering this instruction, the processor switches to kernel mode and jumps to the registered syscall entry path. The kernel completes the context switch (switching the page tables and stack) and then jumps to the specific syscall implementation.

When the syscall finishes, it places the return value in rax and returns. Returning requires another privilege mode switch, reversing everything done on entry: restoring the user page table, stack, and registers.

The following diagram illustrates the sequence of steps required to execute a system call (read in this case).

In the figure:

• User space code sets up arguments for the read system call.

• It invokes the system call using the syscall instruction.

• The instruction switches to kernel mode and enters the syscall entry handler, where the kernel switches to its own page table and stack.

• The kernel then jumps to the implementation of the read system call.

• After returning, the kernel restores the user space page table and stack, then control resumes at the next user instruction.

Now that we have this high-level overview, let’s look inside the Linux kernel’s syscall handler to understand each step in more detail.

Inside the Linux Syscall Handler

When a system call is invoked, the CPU jumps into the kernel’s designated system call handler. The following diagram shows the Linux kernel code for this handler for the x86-64 architecture from the file entry_64.S. In the diagram, you can see the set of steps the kernel needs to perform before it can actually execute the system call. Let’s briefly discuss each of these.

Swapping the GS Register

GS is a segment register in the x86 architecture. In user space it is primarily used for thread-local storage (TLS). In kernel space it is used for implementing per-cpu variables, such as a pointer to the currently executing task. So, the first thing that the kernel does is restore the kernel mode value of the GS register.

Switching to Kernel Page Table and Kernel Stack

The Linux kernel has its own page table with mappings for kernel memory pages. To be able to access its memory it must restore this page table. It does this by calling the SWITCH_TO_KERNEL_CR3 macro.

On x86, the CR3 control register is designated to store the address of the root of the page table. This is why the macro for switching page tables is called SWITCH_TO_KERNEL_CR3.

Separately, the kernel has its own fixed-size stack for executing kernel-side code. At this point the rsp register still points to the user space stack, so the kernel saves it in a scratch space and then restores its own stack pointer from a per-cpu variable.

When returning from the system call, the kernel restores the user page table and stack by reversing these operations. This code is not shown in the diagram but happens right after the “call do_syscall_64” step.

Saving User Space Registers

At this time, the CPU registers still contain the values they had while executing user space code. They will be overwritten when the kernel code executes, to avoid that from happening, the kernel saves the values on the kernel stack. After that it sanitizes those registers for security. All of this can be seen in boxes 3 and 4 in the diagram.

Mitigations Against Speculative Execution Attacks

The next three steps in the code are:

• Enabling IBRS (indirect branch restricted speculation)

• Untraining the return stack buffer

• Clearing the branch history buffer

These are there to mitigate against speculative execution attacks, such as spectre (v1 and v2), and retbleed. Speculative execution is an optimization in modern processors where they predict the outcome of branches in the code and speculatively execute instructions at the predicted path. When done accurately, this significantly improves the performance of the code.

However, vulnerabilities have been found where a malicious user program may train the branch predictor in ways that cause the CPU to speculatively execute along attacker‑chosen paths inside the kernel. While these speculative paths do not change the logical flow of kernel execution, they can leak information through microarchitectural side‑channels such as the cache.

These mitigations prevent user‑controlled branch predictor state from influencing speculative execution in the kernel. But, these also come at a great performance cost. We will revisit these in detail later, when discussing the impact of system calls on branch prediction.

Executing the System Call and Returning Back to User Space

After all of this setup, the kernel finally calls the function do_syscall_64. This is where the actual system call gets invoked. We will not look inside of this function because our focus is on performance impact rather than a walkthrough of kernel code.

Once the system call is done, the do_syscall_64 function returns. The kernel then restores the user space state, including registers, page table, and stack, and returns control back to user space. The following diagram shows the code after the do_syscall_64 call to highlight this part.

Now that we have seen all the code the kernel executes to enter and exit a system call, we are ready to discuss the overheads introduced. There are two categories:

• Direct overhead from the code executed on entry and return.

• Indirect overhead from microarchitectural side-effects (e.g. clearing the branch history buffer and return stack buffer).

The major focus of this article is on discussing the indirect overhead induced due to system calls. But before we go any further, let’s do a quick benchmark to measure the impact of the direct overheads.

Direct Overhead of System Calls

Direct overhead is largely fixed across all system calls, since each system call must perform the same entry and exit steps. We can do a rough measurement of this overhead with a simple benchmark by comparing the number of cycles taken to execute the clock_gettime system call in the kernel versus executing it in the user space.

The clock_gettime system call reads a system clock, such as the realtime clock (seconds since the Unix epoch) or the monotonic clock (seconds since kernel boot). It is very frequently used in software. For example, Java’s System.currentTimeMillis() and Python’s time.time() and time.perf_counter() use it under the hood.

Because system calls are expensive, Linux provides an optimization called vDSO (virtual dynamic shared object). This is a user-space shortcut for selected system calls where the kernel maps the system call's code into each process’s address space so that they can be executed like a normal function call, avoiding kernel entry.

So, we can create a benchmark that measures the time taken to execute clock_gettime in the user space using vDSO and compare it against the time taken inside the kernel using the syscall interface. The following code shows the benchmarking program.

#define _GNU_SOURCE
#include <sys/syscall.h>
#include <unistd.h>
#include <stdint.h>
#include <stdio.h>
#include <time.h>
#include <x86intrin.h>

int main() {
  const int ITERS = 100000;
  uint32_t cpuid;
  struct timespec ts;
  
  // Warm up both syscall and libc versions
  for (int i = 0; i < 10000; i++) {
    syscall(SYS_clock_gettime, CLOCK_MONOTONIC, &ts);
    clock_gettime(CLOCK_MONOTONIC, &ts);
  }

  // Test 1: Direct syscall interface
  _mm_lfence();
  uint64_t start1 = __rdtsc();
  long sink1 = 0;
  for (int i = 0; i < ITERS; i++) {
    long ret = syscall(SYS_clock_gettime, CLOCK_MONOTONIC, &ts);
    sink1 += ret + ts.tv_sec + ts.tv_nsec; // use the results to prevent optimization
  }
  uint64_t end1 = __rdtscp(&cpuid);
  _mm_lfence();

  // Test 2: libc clock_gettime
  _mm_lfence();
  uint64_t start2 = __rdtsc();
  long sink2 = 0;
  for (int i = 0; i < ITERS; i++) {
    int ret = clock_gettime(CLOCK_MONOTONIC, &ts);
    sink2 += ret + ts.tv_sec + ts.tv_nsec; // use the results to prevent optimization
  }
  uint64_t end2 = __rdtscp(&cpuid);
  _mm_lfence();

  // Prevent dead-code removal
  if (sink1 == 42 || sink2 == 42) fprintf(stderr, "x\n");

  double cycles_per_syscall = (double)(end1 - start1) / ITERS;
  double cycles_per_libc = (double)(end2 - start2) / ITERS;
  
  printf("Direct syscall cycles per call ~ %.1f\n", cycles_per_syscall);
  printf("Libc wrapper cycles per call ~ %.1f\n", cycles_per_libc);
  printf("Difference ~ %.1f cycles (%.1f%% %s)\n", 
         cycles_per_libc - cycles_per_syscall,
         100.0 * (cycles_per_libc - cycles_per_syscall) / cycles_per_syscall,
         cycles_per_libc > cycles_per_syscall ? "slower" : "faster");
  
  return 0;
}

A note on rdtsc: Normally, you would use clock_gettime() to measure timings. But here we are benchmarking clock_gettime() itself, so we need something more precise. rdtsc is an x86 instruction that reads the value of a 64‑bit timestamp counter (TSC) in the CPU. This counter ticks at a fixed frequency (e.g. 2.3 GHz on my machine). By measuring its value before and after, we can know how many cycles an operation took.

The program produces the following output on my laptop:

➜ ./clock_gettime_comparison 
Direct syscall cycles per call ~ 1428.8
Libc wrapper cycles per call ~ 157.0
Difference ~ -1271.9 cycles (-89.0% faster)

The vDSO version is an order of magnitude faster, showing how costly the syscall entry/exit path is compared to a plain function call.

We should take this estimate with a grain of salt because in the benchmark we are measuring inside a loop, and the performance of the loop itself can suffer from the indirect side‑effects of entering and exiting the kernel, which is our next topic.

While this benchmark isolates direct overhead, real‑world performance also suffers from indirect costs due to CPU microarchitectural effects. Let’s explore those next.

Indirect Overhead of System Calls

System calls also incur indirect costs, because the kernel’s entry path disturbs the CPU’s microarchitectural state. These side-effects impact the microarchitectural state of the process in the CPU and the loss of this state can introduce transient degradation in the performance of the user space code.

At the microarchitecture level, the CPU implements several optimizations such as instruction pipelining, superscalar execution and branch prediction. These are designed to improve the instruction throughput of the program, i.e., how many instructions the CPU can execute each cycle. A higher throughput means faster program execution.

It can take a few cycles for the CPU to get to a steady state where these optimizations start to pay off, but making system calls can lead to the loss of this state and a drop in the performance of the program.

We will cover the indirect costs of system calls by discussing the different components of the microarchitecture that are impacted, starting from the instruction pipeline, followed by the branch predictor buffers.

Effect on the Instruction Pipeline

We didn’t see any code in the Linux kernel which touches the instruction pipeline, rather this is done by the CPU itself. Before switching to kernel mode, the CPU drains the instruction pipeline to ensure that the user space code does not interfere when the kernel code executes. This impacts the performance of the user space code when the system call returns. To understand how, we need to revisit the basics of instruction pipelining.

CPUs have multiple execution resources, such as registers, execution units, load and store buffers etc. To use all of these effectively, it is necessary that they executes multiple program instructions in parallel, this is made possible through instruction pipelining and superscalar architecture.

Instruction pipelining breaks down the execution of an instruction into several stages, like the assembly pipeline in a factory. An instruction moves from one stage to the next in each CPU cycle, enabling the CPU to start executing one new instruction each cycle.

For example, the following diagram shows a 5-stage pipeline. You can see that it takes five instructions for the pipeline to fill completely, and for the first instruction to retire. After this stage, the pipeline is in a steady state, and it can provide a throughput of one instruction per cycle. This is a very simplistic example, modern x86 processors have much deeper pipelines, e.g. 20-30 cycles.

Modern processors are also superscalar. They have multiple such pipelines to issue and execute multiple new instructions each cycle. For example, a 4-wide processor can start executing up to 4 new instructions each cycle and it can retire up to 4 new instructions each cycle. If such a CPU has a pipeline depth of 20, then it can have up to 80 instructions in flight in a steady state.

This means that the processor is normally busy executing dozens of user-space instructions in parallel. But when a system call occurs, the CPU must first ensure all pending user instructions finish before it can jump into the kernel.

So, when the system call returns back to the user space, you can imagine that the instruction pipeline is almost empty because the CPU did not allow the instructions following syscall to enter the pipeline. At this point the pipeline has to start almost from scratch, and it can again take a while until the pipeline reaches a steady throughput again.

Contrast this with the scenario where no system call occurs: the CPU remains in its steady state, pipelines stay full, and instruction throughput stays high. In other words, a single system call can derail the momentum of dozens of in‑flight instructions.

On x86-64, the syscall instruction is used to execute a system call. The Intel manual has this note about it:

“Instruction ordering: Instructions following a SYSCALL may be fetched from memory before earlier instructions complete execution, but they will not execute (even speculatively) until all instructions prior to the SYSCALL have completed execution (the later instructions may execute before data stored by the earlier instructions have become globally visible).”

This confirms that the CPU drains the pipeline before transferring control to the kernel.

Effect on Branch Prediction

The next major indirect impact system calls have on user space performance is through the clearing of the branch predictor buffers. These can be grouped as three mitigations the kernel applies that we saw in the kernel code above.

• Clearing the branch history buffer

• Untraining the return stack buffer

• Enabling/disabling the IBRS

The first two of these have a profound indirect impact on user code performance. The enabling/disabling of IBRS does not impact user space performance, rather only adds a direct overhead to syscall execution. However, I will discuss this here because logically it goes with the topic of branch prediction. In this section, we will first review branch prediction and then talk about each of these.

Understanding Branch Prediction

Instruction pipelining and superscalar execution enables CPUs to execute multiple instructions in parallel, and they execute these instructions out-of-order.

When the CPU comes across a branching instruction, such as an if condition, it may not know the result of the condition because those set of instructions may still be executing. If the CPU waits for those instructions to finish to know the branch outcome, the pipeline can be stalled for a long time, which means poor performance.

To optimize this, the CPUs come with a feature called the branch predictor that can predict the target address of these branches based on past branching patterns. This enables the CPU to speculatively execute the instruction from the predicted address and stay busy. If the prediction turns out to be correct, then the CPU saves a lot of cycles and instruction throughput remains high.

However, when the prediction is wrong, the CPU has to discard the results of these speculatively executed instructions, flush the instruction pipeline, and fetch the instructions from the right address. This can cost 20-30 cycles on modern CPUs (depending on the depth of the pipeline).

Clearing the Branch History Buffer

We saw in the kernel code that it invokes the macro CLEAR_BRANCH_HISTORY which clears the branch history buffer (BHB).

The BHB is a buffer in the branch predictor that learns the branching history patterns at a global level. This helps the branch predictor predict the outcomes of deeply nested and complex branching patterns more accurately. You can think of it as remembering the last few intersections you passed to better predict where you’ll turn next.

But it can take a while for the BHB to collect enough history for the branch predictor to generate accurate predictions. So, whenever you execute a system call in your code, if the kernel clears the BHB, you lose all that state. As a result, your user space code may experience an increased rate of branch mispredictions after returning from the system call. This can significantly degrade the performance of user space applications.

Note on recent CPUs: This clearance of BHB was added to the kernel as a mitigation against speculative execution attacks, such as Spectre V2. In recent years, CPU vendors have introduced hardware mitigations which obviate the need for the kernel to clear the BHB. For example, the Intel advisory says that if your CPU comes with the "enhanced IBRS" (we discuss IBRS below) feature, then there is no need to clear the BHB. So, not all CPUs suffer degraded performance due to this.

If you want to check whether your kernel clears the BHB, you can check the lscpu output. If you see “BHI SW loop” in the vulnerability section, it means that the kernel clears the BHB during system calls.

Also, if you believe that you will never execute untrusted code, you can manually disable the mitigation through a boot time flag.

Untraining the Return Stack Buffer

Next in the line is untraining of the return stack buffer (RSB). The RSB is another buffer in the branch predictor that is used to predict the return address of function calls.

But why does it need to predict the return address? It again comes down to out-of-order execution. The CPU may want to execute the return instruction even though other instructions of the function may still be executing. At this point, the CPU does not know the return address. The return address is stored on the process’s stack memory, but accessing memory is slow. So, the CPU uses the RSB to predict the return address.

On every function call, the CPU pushes the return address into the RSB. While executing the return instruction, the CPU pops this buffer and jumps to that address. Because this buffer right in the CPU, it is very fast to access.

However, this also led to vulnerabilities such as Retbleed. In this attack, carefully chosen user‑space code could influence how the CPU predicted kernel return addresses, so that the CPU speculatively executed instructions at the wrong place inside the kernel. While this speculative execution did not change the actual kernel logic, it could leak information through side‑channels. To prevent this, the kernel untrains the RSB on entering the kernel.

Untraining the RSB impacts the performance of the user space code when the system call returns because now the RSB does not have the state. Without a trained RSB, the CPU falls back to a slower indirect branch predictor which may have higher chances of making a mistake.

Note on CPUs Impacted: The kernel does not clear the RSB for all the CPU models. The vulnerabilities that require clearing the RSB (retbleed and SRSO) have only been known to impact AMD CPUs. Also, if your CPU has hardware mitigations, such as enhanced IBRS, then the kernel does not perform this (the UNTRAIN_RET macro becomes a noop on such devices).

Again, the kernel allows you to disable the mitigation but do this only when you are sure that you will never run untrusted code.

IBRS Entry and Exit

Finally, let’s talk about indirect branch restricted speculation (IBRS). We saw that the kernel executes IBRS_ENTER on entering the syscall and IBRS_EXIT while returning back. So, what is IBRS and what is its impact on performance?

IBRS is a hardware feature which restricts the indirect branch predictor when executing in kernel mode. Effectively, it prevents the user space training of the indirect branch predictor from having any effect on indirect branch prediction inside the kernel.

Indirect branches are those branches in code where the target address is not part of the instruction but is known only at runtime. A common example is calling through a function pointer in C (e.g., (*fp)()), where the actual target depends on which function the pointer holds at that moment. Another example is a virtual function call in C++ or a jump table generated for a large switch statement. In all these cases, the CPU can use the indirect branch predictor to guess the likely target address based on past branching history.

When the Spectre and related vulnerabilities were found, one of the attack vectors involved tricking the CPU into mispredicting indirect branch targets inside the kernel. By influencing the branch predictor state from user space, attackers could cause the CPU to speculatively execute instructions at unintended locations in the kernel. It could lead to leak of sensitive kernel data through side-channels such as the cache.

The mitigation for this attack is to restrict the indirect branch predictor when executing in kernel mode via the IBRS mechanism. Enabling and disabling IBRS itself doesn’t have any impact on the performance of the user space code, but the act of executing extra instructions to do this during each system call adds overhead.

However, recent CPUs have a feature called enhanced IBRS which automatically enables IBRS when switching to kernel mode. On such devices, the IBRS_ENTER and IBRS_EXIT macros in the kernel become a noop.

Together, these mitigations explain why the indirect cost of system calls can vary significantly across CPU generations and configurations. In practice, this means a single system call can not only drain the pipeline but also leave the branch predictor partially blind, forcing the CPU to relearn patterns and slowing down your code until it recovers. The important point is that the true cost of a system call is not just the handful of instructions executed in the kernel, but also the disruption it causes to the CPU’s optimizations. This makes system calls far more expensive than they look on the surface, and why minimizing them can be such a powerful optimization strategy. However, slowly CPU vendors are adding hardware mitigations which is making these software-based mitigations obsolete and reducing the performance overheads.

Practical Ways to Reduce System Calls

So what can you do as a developer? A few practical ideas:

• Use vDSO: For calls like clock_gettime, prefer the vDSO path to avoid kernel entry.

• Cache cheap values: Some values obtained through system calls rarely change during a program’s lifetime. If you can safely cache them once and reuse, you can avoid repeated system calls.

• Optimize I/O System Calls: There are various strategies and patterns that you can use to optimize I/O related system calls. For example:

  • Prefer buffered I/O instead of raw read/write system calls

  • Use scatter/gather operations like readv/writev to batch multiple buffers

  • If your system allows, use mmap instead of repeated read/write calls.

• Batch operations: Interfaces like io_uring let you submit many I/O requests to a shared queue in user space, which the kernel can then process in batches. This reduces the number of times your program needs to cross into the kernel.

• Push work into the kernel: With eBPF it is increasingly possible to move parts of application logic into the kernel itself. Beyond traditional use cases like packet filtering, newer frameworks let you offload tasks such as policy enforcement, monitoring, and even parts of data processing. In these cases, instead of making repeated system calls, the user program loads small programs into the kernel that run directly when events occur, avoiding crossings altogether.

None of these tricks are magic, but they all follow the same principle: fewer crossings means less disruption. Every time you avoid a system call, you’re saving not just a function call into the kernel, but also the hidden costs of the CPU recovering its state.

Wrapping Up

We’ve gone through a lot of detail for what looks like just a small stretch of kernel code. The point is simple: the cost of a system call goes beyond the small number of instructions that execute in the kernel. It disrupts the CPU’s rhythm by draining pipelines, resetting predictors, and forcing everything to start fresh. That’s why they show up as hot spots in profiles and why people try so hard to avoid them.

The strategies we looked at earlier (vDSO, caching, optimizing I/O, batching with io_uring, and pushing work into the kernel) are all ways to cut down on this disruption. They won’t remove the cost of system calls entirely, but they can make the difference between code that spends most of its time waiting on the kernel and code that keeps the CPU running at full speed.

System calls are the interface to the kernel and the hardware. They are necessary, but they come at a cost. Understanding and managing that cost is a key part of writing faster software.

Share

In this video, I show how adding a single line of code to a slow-running program makes it run 2× faster. You’ll see how this one change helped the compiler arrange instructions in memory so the CPU could fetch them from its micro-op cache instead of decoding them every time, a huge win for hot loops.

On Intel processors, this micro-op cache is known as the Decoded Stream Buffer (DSB). It’s designed specifically to accelerate hot paths in your code by caching pre-decoded instructions, so the CPU can skip the expensive fetch/decode stages entirely. Understanding when and how the DSB kicks in is key to unlocking this kind of speedup.

If you’re curious about controlling the hardware and squeezing out every last ounce of performance, you should watch the video.

Along the way, we’ll cover:

• Measuring performance with Linux perf

• Using Top-Down Microarchitectural Analysis (TMA) to pinpoint hardware bottlenecks

• Understanding what the DSB is and when it’s used

• Forcing the compiler to take advantage of it with code alignment and profile-guided optimization

The result is 2x faster loop and a set of techniques that you can use for debugging and optimizing your own loops.

What’s Next

In this video, I showed how one condition affects whether the processor can use the DSB, and fixing it cut the bottleneck roughly in half. But if you run a top-down analysis again, you’ll still see some DSB stalls. That’s because there are other conditions that also influence DSB usage. In the next video, I’ll dive into one of those remaining conditions and show how to eliminate more of the bottleneck. In the meanwhile, why don’t you experiment and see if you can identify and fix it yourself?

Share

Read more

In algorithm design, we often rely on time complexity to compare solutions. It tells us how the work done by an algorithm grows with the size of its input. But real-world performance also depends on how well the code runs on hardware.

In a previous article, we explored the Iron Law of performance, which states that a program’s performance on the hardware depends on two factors:

• Instruction count: fewer instructions usually mean faster execution.

• Instructions per cycle (IPC): the more instructions a CPU can retire per cycle, the better.

By definition, algorithms with better time complexity tend to do less work. But they don’t always perform better if that work is harder for the CPU. A lower instruction count doesn’t help if each instruction is expensive or slows down IPC.

In this article, we’ll see a concrete example of this tradeoff. We’ll compare three algorithms for computing the greatest common divisor (GCD), study their time complexities, benchmark their real-world performance, and use the Iron Law to understand what’s really going on. As we will see, having a better time complexity is not a guarantee of better performance, a hardware friendly implementation also matters.

Euclid’s Subtraction-based Algorithm for Computing GCD

The first algorithm we will study is Euclid's subtraction-based algorithm for computing the GCD of two integers.

The GCD of two integers a and b is defined as the greatest integer that divides both a and b. For example, the GCD for 12 and 8 is 4. Euclid's subtraction-based algorithm for computing this is shown in the following code block.

gcd(long a, long b)
{
    while (a != b) {
        if (a > b) {
            a -= b;
        } else {
            b -= a;
        }
    }
    return a;
}

The algorithm keeps removing the smaller number from the larger one until both of them become equal. Let’s trace an example with a = 84, b = 18.

As you can see in the above table, the algorithm converges in 7 steps.

Now, let’s think about the worst case time complexity of this algorithm. At each step the algorithm converges towards the GCD value by a step size that is equal to the difference between a and b. The smallest step possible is 1 when either a=1 or b=1. In that case, the algorithm will take as many steps as max(a, b), giving us the worst case time complexity as O(max(a, b))

In simpler terms, the algorithm grows linearly as the difference between a and b grows. For cases, when a and b are nearby, the algorithm converges quickly but when a and b are far apart, it will take a large number of steps.

This subtraction-based approach works, but there's a version that converges faster using division instead of repeated subtraction. Let’s take a look at that.

The Modulo-based Euclidean Algorithm for GCD

A more efficient variation of the Euclidean algorithm replaces repeated subtraction with division, reducing the number of steps required. The following snippet shows the code.

gcd(long a, long b)
{
    while (b != 0) {
        long t = b;
        b = a % b;
        a = t;
    }
    return a;
}

We can trace the same input to see how this version behaves.

In this case, the algorithm converges in just three steps as compared to seven taken by the subtraction-based algorithm. It is easy to see that at each step the algorithm converges towards the solution by performing a division between a and b, which leads to the worst case time complexity of O(log(max(a, b))).

These time complexities give us a theoretical bound of the scale of these algorithms but the actual performance can be measured only by running on real hardware. Let’s run both algorithms on large inputs to see how their time complexity translates to actual performance.

Benchmark #1: Huge Inputs

As an example, let’s compare the performance of the two algorithms on the input a=1000000000 and b=9223372036854775503. Following figure shows the timing and other high-level performance metrics using the Linux perf tool.

The subtraction-based algorithm took 63,34,37,507 steps and ran in 2,230.71 milliseconds, consuming 9.28 billion CPU cycles. It executed 55.35 billion instructions at a rate of 5.96 instructions per cycle.

Now, let’s run the modulo-based algorithm and see how that performs.

As expected, the modulo-based algorithm is dramatically faster, converging in just 22 steps. The CPU executes this in just 0.28 milliseconds, roughly 10,000x faster than the subtraction-based algorithm. It executed only about 1 million instructions and cycles.

This is algorithmic efficiency in action. However, it is not the ultimate truth. The performance of these algorithms also depends on the efficiency of the hardware-level operations. Let’s talk about that for a moment.

Cost of Integer Add vs Integer Division in Hardware

Time complexity tells us how the CPU workload scales with input size. Specifically, the work in the case of these two algorithms refers to subtraction and modulo operations. Inside the CPU, subtraction is performed by the execution unit that performs addition and modulo translates into integer division which is handled by a different execution unit.

So, the performance of these algorithms also depends on the efficiency of these fundamental operations. When talking about the efficiency of CPU instructions like these, we usually care about two aspects:

• Latency: How many cycles does the CPU take to execute that instruction. The lower the latency, the better.

• Throughput: How many of those instructions can the CPU execute each cycle. A processor may be able to execute more than one instruction of a certain type per cycle. Some instructions are pipelined. Even if one takes multiple cycles to finish, the CPU can begin executing another of the same type in the meantime. In addition to that, the processor may have multiple execution ports that can execute that operation in parallel.

On Intel Skylake processors, an add instruction has a latency of 1 cycle and a throughput of 4 operations per cycle because the hardware has four execution ports capable of performing integer addition. Often, the compiler takes advantage of this high throughput by unrolling loops when it notices the loop involves addition operations.

On the other hand, integer division is very expensive. On Intel Skylake, it has a latency of 42-95 cycles. Unlike integer addition, there is only one execution port capable of performing integer division, as a result you can execute only one integer division operation every 24-90 cycles (as per Agner Fog’s optimization manual).

This contrast in the performance of these operations brings the Iron Law into the picture. The subtraction-based algorithm will always execute a higher number of instructions, but it will also have a better IPC. On the other hand, the modulo-based algorithm will execute less number of instructions, but it will have a very poor IPC. Performance depends on the tradeoff between instruction count and IPC. Let’s do another benchmark to see this play out.

Benchmark #2: Small Inputs

In this next benchmark, we use inputs that are close in value to observe how IPC can dominate when instruction counts are similar. The following table summarizes the performance of these two algorithms for the input where a=130000, b=13.

The subtraction-based algorithm executes 9,999 steps as compared to the modulo-based algorithm that converges in a single step. Despite taking 10,000 steps, the subtraction-based algorithm finishes 1 millisecond faster.

If we apply the Iron Law lens, we can see that the subtraction-based algorithm executed a slightly higher number of instructions, but it had a slightly better IPC as well which tipped the performance in its favor.

This isn’t to say algorithmic complexity doesn’t matter, at large scales, it absolutely does. But, a lot of the workload may never hit those scales and the implementation needs to take that into account. For example, many sorting routines use quicksort for large arrays, but switch to insertion sort for smaller sizes (e.g. fewer than 5 elements) because it’s simpler and faster in that regime. Similar strategy can be employed here. For values with smaller gap between them, subtraction-based algorithm can be preferred, while for values with larger difference, modulo-based algorithm can be used.

An alternative to switching between algorithms is to use an implementation that’s inherently hardware-friendly. In 1967, Stein designed such an algorithm that takes advantage of binary representation of integers and leverages bit shift operations that are very fast at the hardware-level. Let’s first understand how it works before comparing performance.

Stein’s Binary Algorithm for GCD

Stein designed this algorithm based on certain observations about GCD computation. These are as follows:

1. gcd(a, b) = gcd(b, a)

2. gcd(0, b) = b, and gcd(a, 0) = a

3. gcd(2a, 2b) = 2 * gcd(a, b), i.e., if a and b are even then we can compute the GCD of their halves and then multiply the result by 2.

4. gcd(a, 2b) = gcd(a, b), i.e., if b is odd then 2 is not a common divisor.

5. gcd(a, b) = gcd(a, b - a) when a and b are odd and a < b.

These observations lead to a recursive algorithm that reduces the inputs until b == 0. However, unlike the modulo-based algorithm, this algorithm can be highly optimized for real hardware. For example, most implementations use the following optimization tricks.

• Instead of recursion, iteration is preferred.

• Every step of the loop has to reduce a and b to odd values. This requires repeated division by 2. It turns out division by 2 can be done by the right bit shift operation which is much faster to perform than division.

• The algorithm needs to check if the numbers are odd or not. Dividing by 2 and checking the remainder is not efficient because division is slow and introduces branches. A more efficient trick is to always ensure that the least significant bit (LSB) of these numbers is 1. This can be done by counting the number of trailing zero bits in the number, and right shifting by that many bits. Most processors have a dedicated instruction to count the number of trailing zero bits, so this is extremely cheap to do.

The following code block shows the C implementation of this algorithm (it uses the GCC builtin __builtin_ctzl to count the number of trailing zero bits).

gcd(long a, long b)
{
    // base conditions
    if (a == 0)
        return b;
    if (b == 0)
        return a;

    // gcd(2^i * a, 2^j * b) = 2^k * gcd(a, b)
    // where k = min(i, j)
    int k = __builtin_ctzl(a | b);

    // make a odd
    a >>= __builtin_ctzl(a);
    while (b != 0) {
        // make b odd
        b >>= __builtin_ctzl(b);
        
        // ensure b > a
        if (a > b) {
            long temp = a;
            a = b;
            b = temp;
        }
        // gcd(a, b) = gcd(a, (b-a)/2))
        // the division by 2 happens in the next iteration
        b = b - a;
    }
    // multiply 2^k back into the final gcd value
    return a << k;
}

At each step of the iteration, the algorithm converges by dividing b by 2, so the worst case time complexity is O(log_2(max(a, b)), which is the similar as the modulo-based algorithm. But this algorithm has the advantage that it uses efficient hardware instructions. This means that the algorithm executes fewer instructions while maintaining a higher IPC, which is a win over the modulo algorithm.

With the theory and hardware considerations in place, let’s now compare all three algorithms side by side.

Benchmark #3: Comparing All Three Algorithms

First, let’s revisit the large-input benchmark: a=1,000,000,000 and b=9,223,372,036,854,775,503. The following table summarizes the key findings.

We see that the binary algorithm performs just as well as the modulo-based algorithm even though it takes a few extra steps to converge.

We can also analyse this using the Iron Law.

• The subtraction algorithm executes 92 billion instructions, which is vastly higher than the ~1 million executed by the other two.

• The IPC of the subtraction algorithm is very high but the high instruction count dominates because of which it takes significant amount of time to finish.

Next, let’s compare the performance for the 2nd input where a=130000, and b=13. The following table shows the numbers. In this case, we see that the binary algorithm matches the performance of the subtraction-based algorithm. This happens because not only it executes fewer instructions but uses efficient instructions, leading to a higher IPC. It strikes the ideal balance from the Iron Law’s perspective.

But comparing the performance on just two inputs is not enough. The following table shows the performance of these algorithms on a benchmark where they compute the GCD for all unique combination of values in the range [1, 100000).

Let’s analyse this for a moment:

• The binary algorithm is the fastest, while the modulo-based algorithm is the slowest despite having the same time complexity as the binary algorithm. This highlights that a superior complexity is not the only thing that gets you performance, the hardware specific implementation is also crucial.

• The modulo-based algorithm executed 13 billion instructions (the lowest) while the subtraction-based algorithm executed 97 billion instructions (the highest). Yet, the subtraction-based algorithm finished 1.5 seconds earlier. It highlights how efficient integer add operation is in processors as compared to division.

• The subtraction-based algorithm had an IPC of 1.16 (the highest), while the modulo-based algorithm had the lowest IPC of 0.15.

• The binary algorithm doesn’t win in instruction count or IPC. But it outperforms in the overall execution time because from the Iron Law’s point of view, it strikes the right balance between the two factors. In fact, if you benchmark these algorithms on a wide range of values, the binary algorithm always gives a consistent performance, while the performance of the other two algorithms can vary depending on the inputs.

Conclusion

Algorithmic time complexity is important, but real-world performance also depends on how well an algorithm maps to the underlying hardware. Doing less work doesn’t help if that work is inefficient or poorly suited to the CPU. The fastest implementations are those that align with the strengths of the hardware, such as low-latency instructions, high IPC, and predictable execution.

Share

Code

All the code used in the analysis behind this article can be found in the GitHub repo linked below.

Check out the code

In this video post we will be discussing the solution to the homework exercise I gave at the end of the post on system calls. The objective of the exercise was as follows:

• Write a toy implementation of the kill command with a few simplifications

  • Hard code the process id to the pid of any running process on your system

  • Hard code the signal number to 9 (for SIGKILL)

  • Exit the program with the return value of the kill system call

If you are yet to try out this exercise for yourself, then I highly recommend that you do it on your own and then come back to this video to verify your solution.

My aim with such exercises is to give you a taste of systems programming along with teaching assembly. With this combined knowledge of assembly and how things work under the hood, you will be well placed to tackle serious projects in higher-level languages such as C, Rust, Go, Java, etc.

As always, feel free to comment here or reach out to me on email if you have any questions.

Share

Read more

“In the beginning, there was a word. Then came the doubleword, and finally the quadword.”

This article is part of our series on x86-64 assembly. So far we have learned to write simple programs that can move some data around and invoke system calls. For the complete list of articles published so far in this series, check out the series overview.

1. Understanding Computer Organization from First Principles
   Bits, memory, and the logic behind modern computing. A gentle dive into the foundations.

2. Binary Arithmetic and Bitwise Operations for Systems Programming
   Signed numbers, two's complement, masking tricks, and bit-level manipulations that matter.

3. The System-Level Foundation of Assembly
   How your code goes from main() to a running process, and where assembly fits in.

4. Building (and Breaking) Your First X86 Assembly Program
   A minimal working program from scratch, with no runtime or C library. Learn by breaking it apart.

5. Debugging X86-64 Assembly with GDB
   Hands-on debugging walkthroughs to inspecting registers, memory, and control flow.

6. Making System Calls in x86-64 Assembly
   How to interact with the operating system directly using syscalls without a C runtime.

I’m also publishing this in the form an ebook (PDF). If you don’t wish to upgrade to a subscription, you can purchase the PDF using the following link. If you are a paid subscriber you can get it at a discount (monthly subs: 20% and annual subs: 50%), please email me for the discounted link.

Purchase Ebook

Introduction

Now that we've written and debugged a few x86-64 assembly programs, it's time to take a closer look at one of the most fundamental pieces of the architecture: the general-purpose registers.

Rather than throwing a table of names and sizes at you, we'll build up a mental model of how these registers evolved, starting from the 8086 and leading up to modern 64-bit hardware. That historical context makes it much easier to understand the naming conventions and relationships, so you're not constantly wondering where things like sil or r8d came from.

The article also includes hands-on exercises to help you understand how values move between registers of different sizes, and to develop an intuition for how partial registers behave. Along the way, we’ll also cover some of the edge cases and architectural quirks. These often overwhelm beginners, but I’ve tried to present them in the right context, so they’re easier to understand and less likely to trip you up.

Registers in the 16-bit Era

The x86 architecture formally began life with the 8086 processor, which was a 16-bit machine. This meant that it had 16-bit wide registers, and its instructions could operate on values up to 16 bits in size.

The general-purpose registers were named after the first four letters of the alphabet: ax, bx, cx, and dx.

8-bit Register Halves

While these registers could work with 16-bit values, there was also a need to handle 8-bit data. Using bitwise masks to access just the higher or lower 8 bits would have been cumbersome and inefficient, requiring extra instructions. To solve this, the 8086 architecture introduced alternate names to refer directly to the upper and lower 8-bit halves of the 16-bit registers.

The naming was logical: replace the "x" in the 16-bit register name with "h" for the high byte or "l" for the low byte. For example, ah refers to the high 8 bits of ax, and al refers to the low 8 bits.

The following diagram shows the full set of general-purpose registers in the 8086, including how the 8-bit halves map onto the 16-bit registers:

Word Size and Instruction Suffixes

If you remember, when we wrote our first x86-64 assembly program, we wrote the following instruction:

movq $32, %rdi

Here, mov is the instruction, and the q suffix stands for "quadword", which in x86-64 means 64 bits.

x86 uses suffixes to indicate operand sizes: 8-bit, 16-bit, 32-bit, and 64-bit. These suffixes evolved along with the architecture, and we'll explore them as we move from 16-bit to 64-bit.

You're right to think that if a quadword is 64 bits, then a word must be 16 bits. The 8086 was a 16-bit processor, and as a result its word size was also 16 bits. In computer architecture, the word size is the number of bits of data that the processor can handle in a single operation. So, the assembly instructions for 8086 used the suffix “w" for 16-bit values.

Hands-on Exercise: Working with 16-bit Registers

Here’s an example that writes two 16-bit values into ax and bx, computes their difference, and exits.

.text

.globl _start
_start:
    # write two 16-bit values into ax and bx
    movw $100, %ax
    movw $58, %bx

    # compute the difference: ax = ax - bx
    subw %bx, %ax

    # exit with status code: 0    
    movq $60, %rax
    # xoring rdi with itself zeroes it
    xorq %rdi, %rdi
    syscall

Try running this inside gdb, and observe the values of the registers ax and bx after each instruction. You can use the following commands to do this:

p (short) $ax 
p (short) $bx

Note About the xor Instruction: In the above program, xorq %rdi, %rdi zeroes out the rdi register. This is a common and efficient trick: XOR-ing a register with itself always results in zero.

Hands-on Exercise: Working with 8-bit Registers

Let’s run a small program that helps you visualize how the ah and al 8-bit halves relate to the full 16-bit ax register.

.text
.globl _start

_start:
    # write a 16-bit value 0x1234 into ax
    movw $0x1234, %ax

    # copy the high 8 bits of ax into bl
    movb %ah, %bl

    # copy the low 8 bits of ax into ch
    movb %al, %ch

    # exit
    movq $60, %rax
    xorq %rdi, %rdi
    syscall

Try this in GDB, and inspect the values of %ax, %bl, and %ch after each instruction. You should see:

• %ax contains 0x1234

• %ah (upper byte of ax) is 0x12 → copied to %bl

• %al (lower byte of ax) is 0x34 → copied to %ch

You can use the following commands to inspect the values of these registers:

p (short) $ax
p (char) $bl
p (char) $ch

Evolution to x86-32 Architecture

Read more

Welcome to my ongoing series on x86-64 assembly programming, designed for programmers who want to peel back the abstraction and understand how code really runs at the machine level.

Why should a software engineer care about assembly? Because understanding what's happening at the lowest level helps you write better code at every level. It sharpens your intuition about performance bottlenecks, compiler behavior, memory usage, and even security. Whether you're debugging a weird bug, chasing a perf regression, or just curious how high-level constructs boil down to machine instructions, assembly is the Rosetta Stone.

We start from first principles, covering bits, memory, and CPU instructions, and gradually build up the skills to read and write real-world assembly programs. Whether you're interested in systems programming, performance tuning, or just curious about what your compiler is really doing under the hood, this series is for you.

Published Posts

1. Understanding Computer Organization from First Principles
   Bits, memory, and the logic behind modern computing. A gentle dive into the foundations.

2. Binary Arithmetic and Bitwise Operations for Systems Programming
   Signed numbers, two's complement, masking tricks, and bit-level manipulations that matter.

3. The System-Level Foundation of Assembly
   How your code goes from main() to a running process, and where assembly fits in.

4. Building (and Breaking) Your First X86 Assembly Program
   A minimal working program from scratch, with no runtime or C library. Learn by breaking it apart.

5. Debugging X86-64 Assembly with GDB
   Hands-on debugging walkthroughs to inspecting registers, memory, and control flow.

6. Making System Calls in x86-64 Assembly
   How to interact with the operating system directly using syscalls without a C runtime.

7. Understanding Registers and Data Movement in x86-64 Assembly

   Systematic coverage of the general-purpose registers in x86-64 architecture and how to move data between them.

8. x86 Addressing Modes, Part 1 — Immediate and Direct Access

   Learn about static data allocation, and accessing memory using immediate and direct access modes. Setting up the foundation for the more advanced addressing modes in the upcoming articles. You will master these two addressing modes by implementing interesting exercises. For immediate addressing mode, you write your own implementation of the cat utility in x86 assembly and for direct memory addressing, you write a benchmarking program.

Upcoming Topics

Here’s a peek at what’s planned for future posts (subject to change based on feedback and curiosity):

• Registers, stack, and calling conventions

• Memory addressing and pointer arithmetic

• Writing loops and conditionals in pure assembly

• Implementing functions and recursion

• A deeper dive into Linux syscalls (file I/O, process management, etc.)

• Mini-project: writing a simple command-line utility

• Capstone: building a minimal web server in assembly

You can subscribe to get new posts as they drop. I’m writing this series with care, making sure each part builds up your intuition as well as your skillset. Feel free to share, comment, or ask questions.

Subscribe now

Share

The same idea applies when writing code in interpreted languages like Python. Sometimes you need to use tricks that help the language’s virtual machine (VM) run your code faster. But just like hardware improves, the Python VM and compiler also keep evolving. As a result, optimizations that once made a difference may no longer matter.

One such optimization trick in Python is to create a local alias for a function you’re calling repeatedly inside a hot loop. Here’s what that looks like:

# Benchmark 1: Calling built-in len directly
def test_builtin_global(lst: list):
    for _ in range(1_000_000):
        len(lst)

# Benchmark 2: Aliasing built-in len to a local variable
def test_builtin_local(lst: list):
    l = len
    for _ in range(1_000_000):
        l(lst)

This trick works because of how Python resolves variable names. Creating a local alias replaces a global lookup with a local one, which is much faster in CPython. But is it still worth doing?

I benchmarked this code across recent Python releases, and the results suggest that the answer is: not really. So what changed?

To answer that, we’ll need to dig into how Python resolves names during execution, and how that behavior has evolved in recent versions. In particular, we’ll explore:

• Why this trick worked in earlier versions of Python

• What changed in recent CPython releases to make it mostly obsolete

• Whether there are still edge cases where it helps

  Writing this article took me several days and nights. You can support my work by becoming a paid subscriber. As a paid subscriber you get early access to all articles, exclusive articles and discounted access to courses/books.

Cut Code Review Time & Bugs in Half (Sponsored)

Code reviews are critical but time-consuming. CodeRabbit acts as your AI co-pilot, providing instant Code review comments and potential impacts of every pull request.

Beyond just flagging issues, CodeRabbit provides one-click fix suggestions and lets you define custom code quality rules using AST Grep patterns, catching subtle issues that traditional static analysis tools might miss.

CodeRabbit has so far reviewed more than 10 million PRs, installed on 1 million repositories, and used by 70 thousand Open-source projects. CodeRabbit is free for all open-source repos.

Get Started Today

How Python Resolves Local and Global Names

To understand why this trick made a difference in performance, we need to look at how the Python interpreter resolves variable names, specifically, how it loads locally vs globally scoped objects.

Python uses a stack-based virtual machine. This means it evaluates expressions by pushing operands onto a stack and performing operations by popping those operands off. For example, to evaluate a + b, the interpreter pushes a and b onto the stack, pops them off, performs the addition, and then pushes the result back on.

Function calls work the same way. For a call like len(lst), the interpreter pushes both the function object len and its argument lst onto the stack, then pops and uses them to execute the function.

But from where does the interpreter find and load objects like len or lst?

The interpreter checks three different places when resolving names:

• Locals: A table of locally scoped variables, including function arguments. In CPython, this is implemented as an array (shared with the VM stack). The compiler emits the LOAD_FAST instruction with a precomputed index to retrieve values from this table, which makes local lookups very fast.

• Globals: A dictionary of global variables, including imported modules and functions. Accessing this requires a hash lookup using the variable’s name, which is slower than a local array access.

• Builtins: Functions like len, min, and max. These live in a separate dictionary and are checked last if the name isn’t found in globals.

With that understanding of how name resolution works in CPython, let’s now compare the disassembly of the two versions of our benchmark function.

For a more comprehensive coverage of the CPython virtual machine, check out my article on its internals:

Dissecting Unoptimized Python Bytecode

Let’s take a look at what’s actually happening under the hood. We can use Python’s built-in dis module to view the bytecode generated by our functions. Below is the disassembly of the slower version, the one that calls len directly:

Let’s break down what’s happening in those highlighted instructions:

• LOAD_GLOBAL: This instruction loads the name len from the global scope on the stack. In the disassembly, you’ll see something like LOAD_GLOBAL 3 (NULL + len). That 3 is the argument passed to the instruction. It’s an index into the co_names array, which is a tuple of all names used in the function for global or builtin lookups. So, co_names[3] gives 'len'. The interpreter retrieves the string 'len', hashes it, and performs a dictionary lookup in globals(), falling back to builtins if needed. This multi-step lookup makes LOAD_GLOBAL more expensive than other name resolution instructions. (We will look at how LOAD_GLOBAL is implemented in CPython right after this)

• LOAD_FAST: After loading the function that is to be called, the next thing the interpreter needs to do is to push all the arguments. In this case, len takes only one argument which is the list object. This is done using the LOAD_FAST instruction. It loads the lst object from the local variables using a direct index into an array of local variables, so there’s no hashing or dictionary lookup involved. It’s just a simple array access, which makes it very fast.

• CALL: Next, the interpreter needs to perform the function call. This is done using the CALL instruction. The number after CALL tells the interpreter how many arguments are being passed. So, CALL 1 means one argument is being supplied. To execute the call, the interpreter pops that many arguments from the stack, followed by the function object itself. It then calls the function with those arguments and pushes the return value back onto the stack.

One of the costlier steps here is LOAD_GLOBAL, both in terms of what it does and how it’s implemented. We’ve already seen that it involves looking up a name from the co_names array, hashing it, and checking two dictionaries, globals() and builtins(), before it can push the result onto the stack. All of that makes it noticeably slower than a simple local access.

To understand just how much work it does behind the scenes, let’s now take a look at its actual implementation in CPython.

The code is taken from the file generated_cases.c.h which contains all the opcode implementations. Let’s focus on the highlighted parts that I have numbered.

1. The first highlighted block deals with instruction specialization. As we will see later, the default way of looking up globals is slow because it does not know which global symbol we are trying to load and from where. This information is only available to the interpreter at runtime. Instruction specialization caches this dynamic information and creates a specialized instruction, making future executions of the same code faster. We will circle back to this in a later section. Note that, this optimization was not present before CPython 3.11.

2. The second highlighted block is where the actual global lookup happens. It’s broken into two parts, which I’ve marked with arrows labeled 3 and 4.

3. First, the interpreter needs to figure out which name it’s supposed to look up. The LOAD_GLOBAL instruction receives an argument (oparg), which is an index into the co_names tuple. This is where all global and builtin names used in the function are stored. The interpreter calls the GETITEM macro to fetch the actual name (a string object) using this index.

4. Once the name is retrieved, the interpreter calls _PyEval_LoadGlobalStackRef. This function looks for the name in the globals dictionary first. If it’s not found there, it falls back to the builtins dictionary.

Let’s zoom into this part and see the code for doing this globals and builtins lookup. _PyEval_LoadGlobalStackRef simply delegates to a function called _PyDict_LoadGlobalStackRef, defined in dictobject.c, so let’s directly look at its implementation (shown in the figure below).

Here’s what is happening in this code:

1. First, the function computes the hash of the name which is being looked up. This hash determines the index into the dictionary’s internal hash table.

2. Next, the function checks the globals dictionary.

3. If the name isn’t found in globals, the function falls back to checking the builtins dictionary.

From this entire discussion of global lookups in CPython, few things are worth highlighting:

• The lookup requires a hash computation. This means that when you are repeatedly calling a function in a loop, the runtime is computing the hash each time. That said, string hashes are cached, so the overhead isn’t as bad as it might seem.

• Another thing to note here is that builtins are checked last. So even if you’re calling a builtin function, the runtime still checks globals first and only then builtins. In a hot loop where performance matters, these things matter.

Next, we’ll dissect the disassembly of the code with the optimization in place.

Dissecting Optimized Python Bytecode

Let’s see how using a local alias actually changes the bytecode, and why it makes the optimized version faster. The following figure shows the bytecode disassembly for this version:

Let’s focus on the highlighted instructions that are responsible for the call to l, which is the alias we created for len. The key difference between the unoptimized and this version is that this one uses the LOAD_FAST instruction instead of LOAD_GLOBAL to load the function object onto the stack. So, let’s look at how LOAD_FAST is implemented in CPython (shown in the figure below).

You can see how short and tight this implementation is. It performs a simple array lookup using an index passed to it as argument. Unlike LOAD_GLOBAL, which involves multiple function calls and dictionary lookups, LOAD_FAST doesn’t call anything. It’s just a direct memory access, which makes it extremely fast.

By now, you should have a clear understanding of why this optimization trick works. By creating a local variable for the len builtin, we turned an expensive global lookup into a fast local lookup, which is what makes the performance difference.

But as we saw in the benchmark results, starting with CPython 3.11, this optimization no longer makes a meaningful difference in performance. So, what changed? Let’s see that next.

Inside CPython's Instruction Specialization

CPython 3.11 introduced a major optimization called the specializing adaptive interpreter. It addresses one of the core performance challenges in dynamically typed languages. In such languages, bytecode instructions are type-agnostic, meaning they don’t know what types of objects they will operate on. For example, CPython has a generic instruction called BINARY_OP, which is used for all binary operations like +, -, *, and /. It works with all object types, including ints, strings, lists, and so on. Therefore, the interpreter has to first check object types at runtime and then dispatch to the appropriate function accordingly.

So how does instruction specialization work? When a bytecode instruction is executed for the first time, the interpreter captures some of the runtime information about it, such as the type of the objects, the specific operation being performed, etc. Using that information, it replaces the slow generic instruction with a faster specialized instruction.

Thereafter, whenever the same line of Python code executes again, the interpreter executes the specialized instruction. Inside the specialized instructions, the interpreter always checks that the conditions for specialization still hold true. If the conditions have changed, e.g., the types are no longer the same, then the interpreter deoptimizes and falls back to the slower instruction.

The LOAD_GLOBAL instruction is also a generic instruction. In this case, the interpreter has to do a lot of additional work, such as looking up the name of the symbol, computing the hash, and finally performing lookups in the globals and builtins dictionaries. But once the interpreter sees that you’re accessing a specific builtin, it specializes LOAD_GLOBAL into LOAD_GLOBAL_BUILTIN.

The LOAD_GLOBAL_BUILTIN instruction is optimized to check the builtins dictionary directly, i.e., it skips checking the globals dictionary. It also caches the index of the specific builtin we are trying to lookup, which avoids the hash computation. The result is that it behaves almost like a LOAD_FAST, performing a fast array lookup instead of a costly dictionary access. The following figure shows its implementation.

Let’s break down the highlighted parts:

1. First, the instruction performs some checks to ensure that the conditions for which it specialized the LOAD_GLOBAL instruction to this specialized version still hold true. If the conditions no longer hold, it falls back to the generic LOAD_GLOBAL implementation.

2. After that, it reads the cached index value. This is based on the hash value it computed the last time while executing LOAD_GLOBAL. It means that this instruction is specialized for looking up only the len function.

3. Next is the lookup in the builtins dictionary. This requires first getting access to the keys within the dictionary.

4. From the keys, it gets the list of entries in the internal hash table and looks it up using the cached index value. If it finds an entry, that is the object we were trying to load.

As you can see, an expensive hash table lookup turned into an array lookup using a known index, which is almost the same amount of work as the LOAD_FAST instruction. This is the reason that in the newer CPython releases, we don’t explicitly need to do the kinds of optimizations where we create a local variable for a global function or object. It automatically gets optimized.

But is this optimization of creating a local alias really obsolete? Maybe not. Let me show you another benchmark.

Benchmarking Imported Functions Vs Aliases

Let’s now look at a similar benchmark, this time involving a function from an imported module rather than a builtin. Here’s what the code looks like:

import timeit
import math

# Benchmark 1: Calling math.sin directly
def benchmark_math_qualified():
    for i in range(1000000):
        math.sin(i)

# Benchmark 2: Aliasing math.sin to a local variable
def benchmark_math_alias():
    mysin = math.sin
    for i in range(1000000):
        mysin(i)



# Benchmark 3: Calling sin imported via `from math import sin`
from math import sin
def benchmark_from_import():
    for i in range(1000000):
        sin(i)

There are three benchmarks:

1. benchmark_math_qualified: calls math.sin directly

2. benchmark_math_alias: creates a local alias mysin for math.sin

3. benchmark_from_import: uses sin imported via from math import sin

And the following table shows the results across the recent CPython releases.

In this case, we see that calling math.sin (fully qualified name) is slowest across the releases and creating an alias is fastest. While calling “math.sin” directly has gotten faster in recent Python versions, it still lags behind the alternatives in performance.

The performance gap here comes from how the function object is resolved when using a fully qualified name like math.sin. It turns into a two-level lookup. For example, the following figure shows the disassembly for calling math.sin(10).

Notice that now the interpreter has to execute two instructions to load the function object on the stack: LOAD_GLOBAL followed by LOAD_ATTR. LOAD_GLOBAL loads the math module object on the stack from the global scope. Then, LOAD_ATTR performs a lookup for the sin function in the math module and pushes the function object on the stack.

So, naturally this requires much more work. And the work increases as the number of levels of lookups increase. For example, foo.bar.baz() requires three levels of lookups.

With the recent Python releases, the performance of fully qualified invocation has also improved due to instruction specialization. However, you still have multiple instructions to execute. Whereas in the case of a local alias, the interpreter has to execute a single LOAD_FAST instruction.

Whether it’s worth trading the readability of a fully qualified name, such as math.sin for a small speedup by aliasing it to mysin, depends on your goals. If that part of the code is performance-sensitive, and your profiling shows this line is a bottleneck, then it’s worth considering. Otherwise, readability might matter more.

Wrapping Up

Aliasing global functions to local variables used to be a meaningful optimization. In earlier versions of Python, global lookups involved more overhead, and avoiding them made a measurable difference. With recent improvements in CPython, especially instruction specialization, that gap has narrowed for many cases.

Even so, not all lookups are equal. Accessing functions through a module or a deep attribute chain can still carry overhead. Creating a local alias or using from module import name continues to be effective in those situations.

The larger point is that optimizations don’t last forever. They depend on the details of the language runtime, which keeps evolving. What worked in the past might no longer matter today. If you want performance, it helps to understand how things actually work. That context makes it easier to know which tricks are worth keeping, and which ones you can leave behind in favor of cleaner, simpler code.

Share

In the previous article, we learned to use gdb and used it to debug our crashing program. Eventually, we discovered that after executing the last instruction, the CPU didn’t know the program had ended. It continued reading and executing past the end of the .text section in memory, causing a crash. So, we need some way to make our process exit or stop the execution before that happens. How can we do that?

When we ran our program using the shell command ./false, it was the shell that invoked the fork and execve system calls. These created a new process, loaded our program into memory, and scheduled it for execution on the CPU. Similarly, to terminate our program gracefully, we need to invoke another system call that tells the kernel our process is done.

This system call to exit a process is called “exit”. When we write code in high-level languages, the runtime automatically invokes it after the main function returns. However, when writing freestanding assembly, we need to do it ourselves. For that, we need to learn how to call syscalls from assembly.

In this part, we will:

• Understand what system calls are

• Learn how to invoke them in assembly

• Fix our crashing program step-by-step

• Write a second assembly program using getpid

• Hands-on exercise: a limited version of the kill command

Recap: If you haven’t seen the previous articles in the series, here’s what you have missed:

This article is part of a paid subscriber series.
If you’re enjoying the content, please consider upgrading to a paid plan to unlock the rest of this series. Paid subscribers also get discounted access to courses and books, and the rest of the archive.

Alternatively, you can purchase an ebook version of this series. (If you're already a paid subscriber, email me for a discounted link.)

I Want the PDF

Understanding System Calls

Before we learn how to invoke system calls, let’s first understand why they exist.

Modern operating systems serve two roles: they manage the execution of programs on the CPU and provide safe, unified access to hardware resources like files, memory, and networks. But, application code cannot directly access these hardware features. Why not?

There are three main reasons:

• Hardware abstraction: Devices vary widely in design and interface. The OS hides this complexity by exposing a uniform way to access them. Whether you're reading from an SSD or a magnetic disk, you use the same system call (read), and the OS handles the details.

• Portability: Most modern OSes follow the POSIX standard, which defines a consistent set of system calls. If your application uses only POSIX-compliant syscalls, it can compile and run on any compliant OS with minimal changes.

• Security: If user programs could directly access memory or I/O devices, they could corrupt system state or access other processes’ data. System calls act as a controlled gateway; only kernel code (running in a higher privilege level) is allowed to interact directly with hardware.

This separation of privilege is enforced by the CPU. On x86, the kernel runs in ring 0 (full privilege), while user programs run in ring 3 (restricted mode). All system calls are implemented inside the kernel at ring 0. To invoke them from ring 3, user space programs need a way to trigger a transition into kernel mode using a mechanism provided by the CPU.

Invoking System Calls on x86-64

Read more

“One ring to rule them all, one ring to find them, one ring to bring them all and in the darkness bind them.”

— J.R.R. Tolkien

Software optimizations are messy and often unpredictable. Whether you see a win is not guaranteed, and the reasons are usually unclear. Is there a way to reason about this?

Maybe there is. In this article, I show you one law that explains all low-level code optimizations: when they work, and when they don’t. It’s based on the Iron Law of Performance, a model widely known in the hardware world but relatively obscure in software circles.

What we’ll see is that almost every low-level optimization, whether it's loop unrolling, SIMD vectorization, or branch elimination, ultimately affects just three metrics: the number of instructions executed, the number of cycles needed to execute them, and the duration of a single cycle. The Iron Law ties them together and gives us a unified reasoning model for software performance.

Of course, not all software optimizations fit into this model. Things like algorithmic improvements, contention removal, or language-level tuning (like garbage collection) lie outside its scope. I’m not claiming the Iron Law explains those.

What’s inside:

• The Iron Law of Performance for software

• Loop unrolling: reducing dynamic instructions

• Function inlining: boosting IPC through linearization

• SIMD vectorization: trading instruction count for complexity

• Branch prediction: reducing pipeline flushes

• Cache misses: backend stalls and instruction throughput

• A reasoning framework to guide optimization decisions

Cut Code Review Time & Bugs in Half (Sponsored)

Code reviews are critical but time-consuming. CodeRabbit acts as your AI co-pilot, providing instant Code review comments and potential impacts of every pull request.

Beyond just flagging issues, CodeRabbit provides one-click fix suggestions and lets you define custom code quality rules using AST Grep patterns, catching subtle issues that traditional static analysis tools might miss.

CodeRabbit has so far reviewed more than 10 million PRs, installed on 1 million repositories, and used by 70 thousand Open-source projects. CodeRabbit is free for all open-source repos.

Get Started Today

Background Read:

This article assumes some knowledge of CPU microarchitecture, and optimization techniques such as branch elimination, loop unrolling etc. If you are unfamiliar with these, I recommend the following two articles:

The Iron Law of Performance (Hardware)

First, let’s start by understanding the Iron Law for hardware performance. It is a simple equation that models the performance of the hardware in the context of executing a program. This depends on three factors:

1. Number of instructions executed (also known as the dynamic instruction count)

2. Average number of cycles needed to execute those instructions (cycles per instruction or CPI)

3. Time taken to execute a single CPU cycle (clock cycle time)

The following equation defines the law:

CPU architects use this to analyse how an architectural change impacts the performance of the processor. For example, should they increase the depth of the instruction pipeline?

In a pipelined processor, an instruction moves from one pipeline stage to another in one cycle. Naturally, the cycle time is dependent on the slowest pipeline stage. When you increase the pipeline depth, you breakdown some of the stages into more granular parts, thus reducing the work done in each stage and in turn reducing the cycle time. It means that now the processor can execute more cycles per second.

However, increasing pipeline depth also raises the penalty of cache and branch misses. For example, accessing main memory still takes about 100 ns, which translates to 100 cycles at 1 GHz but doubles to about 200 cycles at 2 GHz when cycle time is halved. Likewise, deepening the pipeline from 15 to 20 stages also increases the branch misprediction penalty from ~15 to ~20 cycles.

These increased latencies and penalties make the average CPI go up as well. So, whether the pipeline depth should be increased and by how much depends on the overall tradeoff. The iron law gives a very simple framework to make these decisions.

When you do low-level software performance optimizations, similar tradeoffs apply. Every optimization affects the program instruction count, cycles per instruction, and sometimes even the CPU clock frequency. So, it makes sense to apply the same model to analyse and reason about software-level optimizations as well. Let’s try to do that in the next few sections.

The Iron Law of Performance for Software

In the context of software, we can slightly tweak the law to the following form:

Here, we’ve replaced CPI (cycles per instruction) with IPC (instructions per cycle). Although they’re mathematical inverses, IPC is more intuitive for software engineers: for example, modern x86 processors can retire up to 4 instructions per cycle, so IPC gives a clearer sense of how close we are to the peak throughput.

We’ve also relaxed the equality to a proportionality. When optimizing software, we’re not looking for exact numbers, rather we’re reasoning about trade-offs.

So, what role do these three terms play in software performance?

• Increasing IPC means the CPU can retire more instructions per cycle, reducing total execution time.

• Lowering the dynamic instruction count means fewer instructions need to be executed overall. In general, this means the CPU needs to do less work and performance should go up.

• Lowering the clock frequency, as sometimes happens with power-hungry instructions (e.g., AVX-512), increases cycle time and harms performance.

We’ll now apply this model to analyze several well-known optimizations: loop unrolling, function inlining, SIMD vectorization, branch elimination, and cache optimizations, and see how each one shifts the Iron Law variables.

Note: These factors themselves are not quite independent from each other. For example, when you reduce the dynamic instruction count of your program, you need to be careful about instruction selection.

As an example, integer additions have a very low latency, whereas integer divisions are expensive. So, if you reduce the instruction count in your program by replacing a high number of integer additions with a small number of integer divisions, your performance may not improve, or it could even degrade. It depends on the overall tradeoff in the decrease in instruction count and the drop in IPC. Whichever factor wins, dictates the performance.

Loop Unrolling

Loop unrolling is a classical optimization. Instead of executing one step of the loop body per iteration, you rewrite the loop to execute multiple steps per iteration. Consider the following loop that computes the sum of an integer array.

int sum = 0;
for (int i = 0; i < n; i++) {
    sum += arr[i];
}

If we unroll this loop four times, it will look like the following:

int sum0 = 0, sum1 = 0, sum2 = 0, sum3 = 0;
int i = 0;

// Process 4 elements at a time
for (; i + 3 < n; i += 4) {
    sum0 += arr[i];
    sum1 += arr[i + 1];
    sum2 += arr[i + 2];
    sum3 += arr[i + 3];
}

// Handle the remainder
int sum = sum0 + sum1 + sum2 + sum3;
for (; i < n; i++) {
    sum += arr[i];
}

Now, let’s reason about how such an optimization can improve the performance of a program and what are the tradeoffs to consider, i.e., in what situations it may not deliver performance improvements.

Note: Usually, you don't have to unroll a loop yourself. The compiler does it when it sees it will deliver better performance. But sometimes it may not do that because it cannot guarantee program correctness due to limited knowledge or constraints about the code. So it is useful to be aware of it.

Impact on Instruction Count

For large n, unrolling reduces the dynamic instruction count. In the example shown above, the loop body executes three instructions: a comparison for loop condition, incrementing the loop counter, and updating the sum. So, the normal loop executes 3n instructions.

A 4 times unrolled loop executes: one loop comparison, one loop index increment and four additions - so six instructions per iteration, and 6n/4 = 1.5n instructions for a vector of size n.

In Iron Law terms, we’ve driven down the Instruction Count by nearly 50 % (for large n), all else equal. This sounds like an obvious performance win, but we need to also look at how this impacts the IPC.

Impact on IPC

Recall from our Iron Law that Performance ∝ IPC / Instruction Count. We’ve already reduced instruction count, so if we can raise IPC (or at least not lower it), net performance improves. Let’s see how unrolling affects IPC.

Increased Instruction Level Parallelism

The main advantage of loop unrolling is the potential increase in the instruction throughput of the program. The processor is capable of executing multiple instructions per cycle, e.g., the modern Intel processors can execute up to 4 instructions every cycle.

However, to achieve that kind of throughput, the processor needs to have enough independent instructions to execute, which is difficult. Usually, instructions have dependencies between them, i.e., the result produced by one instruction is consumed by the next. Such instructions cannot be executed in parallel.

For example, consider the assembly (generated by -O1 flag to GCC) for the body of the normal for loop shown previously (without loop unrolling):

Let me explain what is going on:

• The register edx holds the current value of the sum and at every iteration the value of arr[i] gets added to it.

• The register rax holds the address of the current array element, arr[i]. At the beginning of the loop iteration, the value of arr[i] gets added to the sum value in edx. Then in the 2nd instruction rax gets incremented by 4, which means now rax contains the address of the next array element arr[i + 1].

• Finally, the last two instructions check if we have reached the end of the array, and if not then jump back to the beginning of the loop.

So, for a large array, the CPU is going to be executing these four instructions for a while. Can it execute some of these in parallel so that the loop finishes faster? Not quite. The instructions have dependencies between them that stop the CPU from doing that.

Notice that the first addl instruction that updates the sum in edx depends on the previous iteration's edx and rax values, so the CPU can't issue the next iteration's addl until the previous iteration's addl and addq instructions are finished. In other words, there simply aren't any independent instructions for the CPU to execute in parallel.

Loop unrolling fixes this problem. The following assembly code is the loop body for the unrolled code shown previously.

In the unrolled loop, the compiler assigns each partial sum to its own register (edx, edi, esi, ecx), so each addl instruction uses a different register and memory address. This makes these instructions independent and the CPU can issue and execute those in parallel, reducing the number of cycles needed to finish the loop, and improving the IPC.

In Iron Law terms, we’ve increased IPC from roughly 1.0 (due to dependencies) to perhaps ~3.0 or higher, depending on execution port availability. Combined with a 50 % drop in instruction count, that yields a significant net gain.

Note: How many of these add instructions will actually execute in parallel depends on how many functional units are there in the CPU to perform integer addition. So how much unrolling to do also depends on what kinds of instructions are there to execute.

However, it isn’t all rosy and shiny. Loop unrolling can also hamper the IPC in other ways. Let’s see how.

Register Spills Due to Increased Loop Body Size

Unrolling the loop creates many local variables and increases the demand for registers. When unrolled too many times, or when unrolling a large complicated loop, it can result in register spills. It means that for some variables the compiler will have to use the stack when it runs out of registers.

When register spills happen, the instructions that read data from the stack instead of registers take longer to finish. While a value can be read from a register within a single cycle, reading from stack can take 3-4 cycles (assuming an L1 cache hit).

So, the operations that could be done in a single cycle will now take several cycles due to memory access. In such situations, if the CPU doesn’t have other instructions to execute, it will sit idle and waste resources. This increases the average cycles per instruction and lowers the IPC.

In Iron Law terms, register spills reduce IPC, which can partially or completely negate the instruction count reduction. You must weigh these together.

Note: It is not guaranteed that a register spill will necessarily drop the IPC, because sometimes the compiler can schedule the instructions better to keep the CPU busy while other instructions are stalled on memory access. But it is a tradeoff that you risk introducing when being too aggressive with this optimization.

Instruction Cache Pressure Due to Increased Loop Body Size

Another potential impact of unrolling the loop is the increased code footprint that can cause pressure on instruction cache. These days the instruction caches are large enough that unrolling a loop will not result in cache misses for the loop itself, but in larger systems where there is existing pressure on the instruction cache, this may cause eviction of other instructions.

For example, most x86 cores have a 32-64 KB L1 I-cache. For a loop body consisting of four instructions of 4 bytes each, unrolling it 4 times may increase the code size by ~64 bytes but that is negligible.

So, in general it is not a huge concern for high-end CPUs. But we still need to be aware of the tradeoff from Iron Law's perspective because increased instruction cache misses lower how fast the CPU frontend can send instructions to the backend for execution, thus lowering the IPC.

Loop Unrolling from the Lens of the Iron Law

Below is a summary of trade-offs when viewing loop unrolling through the Iron Law.

Function Inlining

Next, let’s see how function inlining shifts the balance between instruction count and IPC. It is a simple optimization that the compiler routinely performs where it inlines a function call. It means that it replaces the function call with the body of the function being called to avoid the overhead of the function call. Let’s understand with an example.

Consider the following C function and its assembly (generated by GCC with -O1)

Function calls incur hidden costs beyond the core logic. When this function executes, several additional steps occur:

• Stack Frame Setup: A stack frame needs to be set up for the function to manage the function local data on the stack. At the minimum, it requires saving the rbp register on the stack and then copying the current value of the rsp register (the stack pointer) into rbp. So, at least two instructions. These days the compilers optimize this away if they notice that the function doesn’t use the stack, but it is not guaranteed.

• Saving and Restoring Callee Registers: As per the System V AMD64 calling convention, certain registers are required to be saved by the callee function if it needs to use them. This is needed because those registers might be in use by the caller and if the callee doesn’t preserve the previous values, then the caller’s state will be corrupted. So, sometimes you will notice code to save and restore these registers as well. In the case of the code shown above, the function is simple enough that it doesn’t need to do this. But it is also a potential cost of calling functions.

• Destroying Stack Frame: As the function returns, it needs to destroy the stack frame and restore the stack in the state as it was before the call.This again incurs extra set of instructions as you can see in the assembly.

• Function Return: Finally, the ret instruction is required to return the control from the function back to the caller.

Apart from the extra work inside the called function, calling the function also requires extra instructions. The following figure shows a main function calling compute, and on the right hand side you can see the assembly code.

So, what are the overheads while making the function call?

• Saving/Restoring Caller Saved Registers: The Sys V AMD64 ABI defines certain registers as caller saved registers, meaning that the caller needs to save these registers on the stack before making the function call so that the callee function can use these registers freely. In the above code for main, you can see it saving the values of rax and rdx on the stack using the push instruction. In this case, the compiler does not care about restoring them back, but usually you also need to restore them back after the function call returns by a corresponding pop instruction. So, you have two extra instructions per saved register.

• Setting up Function Arguments: Before invoking the function call the caller needs to set up the function call arguments. The Sys V AMD64 ABI designates certain registers in which these arguments can be passed. In the assembly code, I’ve highlighted the instructions which set up the registers.

• Calling the Function: Calling the function requires an extra call instruction. Compared to everything else this looks like a small cost, but nevertheless, when executing a function very frequently, it adds up.

So, when you inline a function, you save all this extra work that the CPU needs to do each time the function is called. The following figure shows a version of the program where the compute function has been inlined in main.

As you can see, inlining has streamlined the entire flow. The extra instructions are gone and only the core logic of the inlined function remains.

Now, it may look like an obvious performance win but let’s analyse from the perspective of the iron law.

Impact on Program Instruction Count

Function inlining reduces the dynamic instruction count of the program (number of instructions executed) in two ways. One is directly due to avoiding the function call overhead, and second is by compiler optimizations that get unlocked after inlining.

Direct Reduction in Instruction Count

• call/ret instruction elimination

• stack frame setup/teardown elimination

• function argument handling elimination

• register save/restore elimination

Even if we conservatively assume a saving of 5 instructions for inlining a function that is called one million times, we save the CPU from executing 5 million extra instruction. This also gives the CPU cycles to execute other instructions, improving the overall IPC.

Context-Sensitive Optimizations

Apart from eliminating function call overhead, inlining gives the compiler opportunity to do further optimizations of the inlined code because it has more information about the context in which the function was being called.

For example consider the following code (source).

int sat_div(int num, int denom) {
  if (denom == 0) {
    return (num > 0) ? INT_MAX : INT_MIN;
  }
  return num / denom;
}

int foo(int a) {
  int b = sat_div(a, 3);
  return b;
}

After inlining, the sat_div function into foo, the compiler may simplify it to following. It can do that because it knows that the 2nd parameter is always 3 when called from foo.

int foo(int a) {
  // The generated code for this looks confusing because
  // the compiler has turned a division into a multiplication.
  return a / 3;
}

These kind of optimizations may not always be possible, but it is a potential positive outcome that can further reduce the number of instructions that the CPU needs to execute.

Impact on IPC

Again, the impact of function inlining on the program IPC is not direct, but via many indirect factors. Let’s see how.

Increased Register Pressure

Inlining large functions with many variables can increase the register pressure and cause a potential spill onto the stack. The compiler needs to make a complex decision considering things like register pressure, function size and other factors. So, usually it may not inline a function if there is going to be a spill. But, if you are manually inlining a function, or forcing the compiler to inline, then it is a potential factor that may impact the IPC of your program.

Increased Code Size

Inlining functions results in a large code footprint because you are making copies of the function everywhere it is called. This increases instruction cache pressure and can cause increased cache misses. Frequent instruction cache misses can starve the CPU backend for new instructions to execute, causing the IPC to drop.

Instruction Level Parallelism

Inlining functions can increase instruction level parallelism because it eliminates the branch introduced in the code due to the function call. It gives the CPU a larger window of instructions to analyze that may enable it to find more work to do in parallel, thus improving the IPC.

Function Inlining from the Lens of Iron Law

Again, we can see that whether or not performance improvements are seen from function inlining depends on several factors. But eventually, all of these factors result in two metrics: the dynamic instruction count, and overall IPC of the program. The following table summarizes these tradeoffs.

With loop unrolling and inlining covered, next we’ll see how SIMD vectorization affects instruction count, IPC, and even clock frequency.

SIMD Vectorization

Single instruction multiple data (SIMD) is a widely used optimization technique when the algorithm performs the same operation on multiple data elements. This is particularly applicable in numeric computing, image processing and similar domains. It improves the performance by significantly improving the IPC and lowering the dynamic instruction count, because the CPU is able to do more work in less number of instructions.

As an example, consider the following function and its assembly:

The function is performing vector addition. I’ve used the -O1 flag which prevents the compiler from vectorizing it. The label .L3 contains the loop body. It executes 4 instructions per element of the vectors. If we manually vectorize the code, or use the -O3 optimization flag, we get the following assembly output which uses SIMD instructions.

Here, the compiler has generated SIMD instructions like vmovups and vaddps which can process 8 float values (256 bits) in a single instruction. It means that the SIMD version of the loop can process 8 elements in just 3 instructions. Compare it to the non-vectorized version shown previously that was executing 4 instructions per array element.

Now, let’s analyze how SIMD vectorization impacts the program performance from the lens of the Iron Law.

Impact on Instruction Count

If you look at the vectorized assembly output above, it may appear as if the number of instructions have significantly increased. However, what matters is how many instructions that the CPU eventually executes.

Typically, for large datasets, the number of instructions executed by the CPU drops significantly when SIMD is used, because each instruction operates on multiple data points simultaneously.

In the example above, the scalar version of the loop executed roughly four instructions per element: two loads, one addition, and one store. In contrast, the vectorized version performed the same work using just three SIMD instructions, each operating on eight elements at once. This reduces the instruction count per element from 4 to 0.375, yielding a theoretical 10.66× reduction in instruction count within the vectorized loop.

So, in general, vectorization results in a massive decrease in the instruction count executed by the CPU.

Impact on IPC

Let’s analyze how SIMD instructions impact the IPC of programs.

Increased Code Size and its Effects on Instruction Cache

Vectorization increases the overall code footprint. One reason is due to the complexity of the vectorization algorithm and extra instructions for processing the left over elements.

Apart from that, on x86, SIMD instructions tend to be longer than scalar ones (due to decades of backward-compatible extensions). When code is heavily vectorized, this increased instruction size leads to a larger static code footprint.

Also, some SIMD instructions have higher latency and can take a few cycles to produce their results. This means that the dependent instructions have to wait longer and the overall instruction throughput is lower. The trick to overcome this is to manually unroll the loop in code. Again, this means even larger code size.

In summary, vectorization inflates the static code footprint. This expanded footprint can evict hot code from the L1 instruction cache, causing front-end stalls and lowering IPC.

Note: While most tight vector loops will easily fit within a 32–64 KB I-cache, it is a tradeoff worth being aware of.

Impact on Instruction Fetch and Decode

On x86, many SIMD instructions, especially AVX and AVX-512, are wider and more complex to decode than typical scalar instructions. This reduces how many instructions the CPU frontend can fetch and decode per cycle, lowering IPC.

Modern x86 processors can fetch 16 bytes and decode up to 4 instructions per cycle. With 3-byte scalar instructions, that’s enough bandwidth for peak throughput. But with 7-byte AVX-512 instructions, only 2 can be fetched per cycle, and decode throughput can drop to 1–2 instructions per cycle.

To mitigate this, CPUs use a μop cache to store decoded instructions. During loop execution, this often hides the decode bottleneck. But for small vector sizes, where the loop runs only briefly, this overhead can dominate and the drop in IPC can negate any performance gains from vectorization.

Register Pressure

Vectorized code usually results in a much larger set of instructions. This results in more general-purpose registers being used as part of the loop being vectorized, increasing the register pressure.

Apart from that, sometimes the loop itself has to be unrolled to take into account the higher latency of some SIMD instructions. This requires using more SIMD registers, which puts pressure on the limited number of SIMD registers.

If vectorization results in register spills, frequent memory accesses can decrease the overall IPC due to the stalls while waiting for memory loads to return. If this becomes a dominant factor, it may eat up the gains provided by SIMD instructions.

Note: With AVX-512 having 32 registers, the chances of a spill are slim but it is worth mentioning this factor for a full coverage of all the tradeoffs.

Impact on Clock Cycle Time

Remember that the Iron Law has three terms: dynamic instruction count, IPC and the CPU clock cycle time. So far all the optimizations we discussed only affected the first two factors, but SIMD instructions impact the clock frequency as well.

On x86, certain SIMD instructions (AVX2 and AVX-512) cause the CPU clock frequency to dynamically scale down to keep the power consumption and temperature in control. In other words, it increases the clock cycle time (negatively impacting the performance).

From Iron Law's perspective, vectorization lowers the dynamic instruction count, but also increases the clock cycle time. It results in an interesting tradeoff where the wins are not obvious.

SIMD wins are clearest when most work is vectorized, but in mixed workloads the reduced clock speed can hurt performance. For example, Cloudflare saw a 10% drop in web-server throughput (requests served per second) when using vectorized hashing algorithms. They found that the CPU spent only 2.5% of the time in vectorized code, and the remaining time in scalar execution. In this case, the increased cycle times outweighed the SIMD gains—Iron Law in action.

Analyzing SIMD from the Lens of Iron Law

In summary, SIMD vectorization can slash instruction count and boost IPC, but it also increases cache footprint, register pressure, and may slow the clock. Here’s how these factors map to the Iron Law metrics:

Optimizing Branch Mispredictions

Branch mispredictions are a common performance bottleneck. When the CPU predicts a branch incorrectly, it must discard the speculatively executed instructions and start fetching from the correct address. This flush costs around 15 to 20 cycles, and sometimes more depending on the pipeline depth. These stalls hurt the CPU’s instruction throughput and directly lower the IPC.

To understand why prediction is needed in the first place, consider how the CPU executes instructions. The backend, which performs the actual computation, is fast and parallel. But it depends entirely on the frontend to deliver a steady stream of decoded instructions. This pipeline works well when code is linear. But when there is a conditional branch, the next instruction depends on the result of a previous comparison instruction. Waiting for that result introduces bubbles in the pipeline and wastes cycles.

To avoid this delay, the CPU speculatively picks a direction using its branch predictor and fetches instructions along that path. If the guess is correct, the pipeline stays full and performance remains high. If the guess is wrong, the CPU has to roll back the speculative work, fetch the correct instructions, and refill the pipeline. This flushing not only delays execution but also wastes frontend bandwidth.

Even though modern branch predictors are highly accurate, often over 95 percent, some branches are unpredictable by nature. Others suffer because the predictor's limited buffer space gets overloaded in large, complex programs.

To see how costly this can be, imagine a loop with one million iterations and a branch that is mispredicted five percent of the time. At a penalty of 20 cycles per miss, this results in one million wasted cycles. That is enough to wipe out the benefit of most other optimizations.

Let’s now analyze how optimizing branches affects performance using the Iron Law framework.

Impact on Instruction Count

The exact impact on the dynamic instruction count depends on the specific optimization and how it changes the code execution. For example, you can reorder the conditions in a loop so that the most predictable and frequently executed branch is at the beginning, thus resulting in reduced branch misses. In this case, the overall number of executed instructions may drop.

Sometimes you may replace a branch with branchless logic using bitwise operations or other techniques. This usually results in more instructions but eliminates the branch prediction. So, you may end up with a higher number of instructions executed but save the misprediction penalties.

The point is that whether or not you see a significant improvement in performance largely depends on the IPC factor. Let’s analyze that.

Impact on IPC

Branch optimizations can impact the IPC in a variety of ways.

Reduced Pipeline Stalls and Flushes

When done well, reduced branch misses result in reduction in pipeline stalls and flushes, resulting in improved instruction delivery to the backend and an improved IPC.

Impact on ILP

Another aspect to consider with branch optimizations is their impact on instruction level parallelism.

Often, branchless implementations require more instructions than their branching counterparts and introduce serial dependencies. For example, consider the following function for conditionally swapping two values.

And the following figure shows a branchless way of implementing the same function along with its assembly output.

You can clearly see that the branchless version has more instructions.

But, the problematic part is that the branchless version results in dependent instructions, which lowers the ILP. In the example above, the compiler has used the cmov instructions that conditionally copy a value depending on the result of a condition. These instructions sidestep the branch predictor, but they introduce data dependencies. Any subsequent instruction that depends on a result produced by a prior cmov instruction has to wait until the cmov instruction has finished. This lowers the potential ILP.

Following figure shows another example of a branchless function to compute max of two integers. Again, you see that most instructions read and write the eax register, creating a dependency chain among them. The CPU will have to execute these sequentially, resulting in significantly lower ILP.

So, the moral of the story is that branchless code comes at the cost of reduced ILP (and thus lower IPC). It is only worth doing when the branch predictor is exhibiting a high number of misses for that code. Otherwise, optimizing a branch which is highly predictable can backfire because the drop in IPC will dominate everything else.

Analyzing Branch Optimization from the Lens of Iron Law

Branch optimization is a tricky optimization to do but as far as its performance gains are concerned, they are relatively easy to reason about from the lens of iron law. Here’s a concise overview:

Analyzing Cache Misses from the Lens of Iron Law

The final optimization we’ll discuss is minimizing cache misses. I’ll keep this section brief. The goal is to complete the picture and demonstrate that the Iron Law applies to almost every low-level optimization you might attempt.

Data cache misses can significantly reduce IPC. When an instruction misses the cache, it must wait hundreds of cycles for the data to arrive from main memory. During this time, it occupies CPU resources such as a reservation station slot, a reorder buffer slot, and a physical register. Any instructions that depend on the result of this instruction cannot proceed either. They remain in the reorder buffer until the data becomes available.

If things like reorder buffer, reservation stations are new to you, I suggest reading my article on the microarchitecture of SMT processors which touches on these details.

When enough of these long-latency instructions accumulate, they create backpressure in the backend. Reservation stations fill up, the reorder buffer approaches capacity, and eventually the frontend is forced to stop issuing new instructions. This limits parallelism and reduces the number of instructions executed per cycle.

Optimizations like structure padding, blocking, data layout transformations, and software prefetching aim to reduce miss rates and improve IPC. However, these often increase instruction count (both static and dynamic) because of added pointer arithmetic, bounds checks, or loop restructuring. The tradeoff is clear: does the reduction in stalls outweigh the overhead of extra instructions?

We will not go into further detail here, but the pattern should feel familiar by now.

Conclusion

We began the article with a bold claim: that one law can explain all low-level code optimizations. After walking through multiple examples, that claim feels much less dramatic.

Most low-level optimizations shift one or more of the Iron Law variables: instruction count, IPC, or clock-cycle time. We often focus on isolated effects, like reducing cache misses or branches, and get confused when performance doesn’t improve. But that confusion disappears once we look at the bigger picture.

The Iron Law gives us a way to step back and see the full picture. It helps us reason about trade-offs clearly, without relying on guesswork. When combined with tools like perf, which show instruction counts, IPC, and backend stalls directly, it becomes easier to understand not just whether something changed, but whether it changed in the right direction.

This model doesn’t apply to every kind of optimization. It won’t help with algorithmic improvements or garbage collection tuning. But for the kind of low-level performance work that many developers struggle to reason about, it gives a clear lens.

So next time you’re tuning code, ask yourself which of the three metrics you’re moving. You might find that one law really does explain more than you expected.

Share

In this part, we'll use gdb to investigate what exactly went wrong. You'll learn how to step through your program instruction by instruction, inspect memory and register values, and get a better sense of how the CPU executes your code. No new assembly instructions yet, just the tools to understand what you're building.

If you haven’t seen the previous articles in the series, here’s what you have missed:

This article is part of a paid subscriber series.
If you’re enjoying the content, please consider upgrading to a paid plan to unlock the rest of this series. Paid subscribers also get access to recordings of past live sessions, early and discounted access to courses and books, and more.

Alternatively, you can purchase an ebook version of this series. (If you're already a paid subscriber, email me for a discounted link.)

I Want the PDF

Quick Recap

Let’s start with a short recap of where we left things in the previous article.

We learned enough assembly to write the following program

# create the text section
.section .text

# mark the _start label as globally visible for linking
.globl _start 

_start:
  movq $32, %rdi
  movq $10, %rsi
  addq %rsi, %rdi

Then we assembled and ran it as follows:

as -o false.o false.s
ld -o false false.o
./false
Segmentation fault (core dumped)

The program is mysteriously crashing, and our goal is to identify why. We will learn to use the debugger (gdb) to do this.

Assembling with Debug Symbols

To debug the program using gdb, the program binary needs to include the debug symbol table so that the debugger is able to show us the source code as we step through it. By default, the assembler and compiler don’t include the debug symbols in the binary because it increases the size of the program binary that can slow down the program startup time.

So first, we need to reassemble our program with debug symbols. To do that with the GNU assembler, we need to use the gstabs option, as shown below.

as -o false.o -gstabs false.s

The linking step remains the same. If the object file produced by the assembler contains the debug symbol tables, the linker will put them in the final executable without us asking it.

ld -o false false.oHere's the program we wrote earlier that was crashing:

Debugging with GDB

The simplest way to debug a program with gdb is to start the program with gdb as shown below.

gdb ./false

Once you do that, you will see the gdb prompt, like the screenshot below.

The (gdb) thing is the GDB prompt, which is where we enter the debugging commands. The first command we will learn to use is to set up a breakpoint.

Setting and Hitting Breakpoints

Read more

Recap from the previous article: In the last couple of articles, we built a simple but complete mental model of how a basic computer executes instructions. We explored how an ALU performs arithmetic operations, how registers serve as fast-access storage, and how the control unit fetches and executes instructions stored in memory. We also introduced the structure of assembly programs, the use of labels, and how data and instructions are laid out in different sections like .text and .data. That article provided the conceptual foundation we need to now dive into real X86-64 assembly code.

This article is part of my series on the basics of X86-64 assembly programming. Until now, we have been working mostly with ideas. We talked about what it means for a computer to execute a program, how computation is carried out by hardware, and how memory is laid out to store data and instructions. We have seen snippets of assembly here and there, but we haven’t written a full program yet. That changes now.

In this article, we will write our first complete (well, almost) assembly program. It won’t do anything exciting, but that’s the point. Like “Hello, world” in high-level languages, this program is just a vehicle to help us understand the mechanics of how an assembly program is written, assembled, linked, and executed. Along the way, we’ll revisit some of the concepts we’ve discussed before and see how they manifest in actual code.

If you haven’t read the previous articles in this series, here’s what you have missed:

This article is part of a paid subscriber series.
If you’re enjoying the content, please consider upgrading to a paid plan to unlock the rest of this series. Paid subscribers also get access to recordings of past live sessions, early and discounted access to courses and books, and more.

Alternatively, you can purchase an ebook version of this series. (If you're already a paid subscriber, email me for a discounted link.)

I Want the PDF

Read more

We wrapped up the X86-64 assembly course last week, and I’ll be sharing notes from the sessions here as a series of articles. While the live sessions covered much more ground, I think you’ll find these write-ups valuable in their own right. I’ll be publishing them gradually over the next few weeks.

I’ll announce the next run of the course soon. Paid subscribers get early access and a discount, so upgrade now if you’d like to reserve your spot.

Subscribe now

I’m also compiling this article series into a cleanly typeset PDF edition, available for purchase below. If you're a paid subscriber, email me for a discount code.

Get the Ebook

In the previous article, we looked at how a CPU executes instructions by fetching them from memory, decoding them, executing them, and repeating the cycle. This gave us a foundational understanding of how the hardware runs a program.

Now we can start writing some assembly programs, but jumping in immediately would mean skipping a few important abstraction layers, and that might leave some gaps in our understanding.

Specifically, an assembly program is translated into an executable binary by the assembler and the linker. That binary is then loaded into memory by the operating system so that the CPU can begin executing it. So there’s quite a bit that happens between writing assembly and running it on real hardware.

The operating system must load the executable into memory in a layout that follows the hardware’s execution model: instructions must be contiguous in memory, and instructions and data must be kept separate. To enable this layout, the executable binary must reflect this structure. Since the binary is generated from the assembly source files, the assembly program must also follow a structure that matches. All of this is linked.

In this article, we’ll trace how the structure of assembly programs is shaped by the expectations of the hardware’s execution model. We’ll follow this causal path through the operating system and the ELF file format, and explain why assembly programs are written in terms of sections and labels, like the one shown in the figure below (don’t worry if it looks alien, I promise that by the end of the article you will understand what each line of this code is doing).

By the end of the article, we will learn the following:

• How the operating system structures a process's memory layout and why that structure is required

• How ELF executables are structured and why they follow that layout

• What sections and labels are in assembly programming and why you need to use them

Recap: Hardware's Instruction Execution Model

Let's start by doing a quick recap of the hardware's instruction execution cycle which we covered in quite detail in the previous article. This background is the foundation for everything we will learn in this article.

• The processor has a special register called the instruction pointer register that contains the address of the instruction that the processor has to execute next.

• The processor also has a component called the control unit that is responsible for orchestrating the execution of program instructions on the hardware. The control unit begins by fetching the instruction located at the memory address held in the instruction pointer register

• The control unit then decodes the instruction to identify the opcode and the operands. For example, the opcode may be add and the operands could be the registers R8 and R9

• Next,the control unit sends control signals to the register file and execution units to execute the instruction.

• After this, the instruction pointer register gets incremented by the size of the current instruction so that it now has the address of the next instruction.

• And this cycle repeats.

This is the hardware execution model. To enable hardware to execute your code, you must load your program’s instructions and data into memory and then update the instruction pointer with the address of the first instruction of your program.

Fortunately, we don’t need to do this ourselves, the operating system does this for us. And how the operating system does this is directly shaped by the hardware’s expectations. Let’s see how the operating system sets up a new process for execution.

Process Setup and Memory Layout

The hardware instruction execution model puts certain requirements on the OS when creating a new process:

• The instruction pointer must contain the address of the first instruction of the program in memory

• The subsequent instructions must be stored contiguously in memory. Because after the first instruction, the hardware simply increments the instruction pointer to find the address of the next one. This model requires that all instructions must be placed next to each other in memory.

  • Figure-2 shows this more visually, where you can see how by simply incrementing the address in the instruction pointer the CPU can advance through your program to execute it.

• A third, often unstated, requirement comes from security concerns: code and data must be kept separate in memory. The hardware itself doesn’t distinguish between them, so a malicious program could modify its own data to insert harmful instructions. If the CPU then executes that data as code, it could lead to serious exploits. To prevent this, it's essential that:

  • Instructions are stored in a memory region with read and execute permissions, but not write

  • Data is stored in a separate region with read and write permissions, but not execute — except in special cases like JIT compilers

So, when the OS creates a new process, it organizes the address space layout of the process to satisfy these hardware-level requirements. The following diagram shows this layout and you can clearly see it is split into distinct memory segments, such as .text, and .data which is where program instructions and static data are stored.

You may have seen this memory layout of a process in many articles and books, but not many explain the underlying reason behind it. But now you know!

Each of these segments appear as part of a single virtual address space, but at the physical level they may be mapped to different regions of the physical memory. Each segment also has the appropriate protection bits set to ensure that the hardware doesn't end up reading and executing instructions from one of the other segments.

But creating these segments is not enough; they also need to be populated with data. For this the operating system loads the program's executable binary into memory and loads the .text segment with all the code, and the .data segment with the statically initialized data.

However, to do this efficiently, the assembler and linker must generate the executable in a format that supports fast loading. On Unix like systems, this format is the executable and linkable format (ELF), and it is designed to support fast loading of program data into the process memory. Let’s see how this format looks like from the inside.

Understanding the ELF Executable Format

Read more

In our previous article, we explored how computers work from transistors up to program execution. We saw how digital circuits built from logic gates perform calculations using binary data, and how the ALU executes operations on this binary representation.

Now we’ll dive deeper the binary number system. When writing assembly code, you’ll directly manipulate bits in registers, perform calculations at the bit level, and for that you need to understand exactly how the processor interprets the patterns of 1s and 0s.

This article covers four key areas:

1. Number systems: How binary and hexadecimal work, and why we use them in low-level programming

2. Binary arithmetic: How computers add and subtract, and detect conditions like overflow

3. Two’s complement: How the hardware represents negative numbers

4. Bitwise operations: bit manipulation techniques used throughout systems programming

These concepts appear repeatedly in assembly programming, from register manipulation to optimized algorithms. By developing an intuition for binary operations, you’ll gain deeper insight into how processors work and how to write efficient assembly code.

Read it in PDF Form

This article is part of the material I’m developing for my X86 assembly course, that I am also putting together in a PDF form. If you are a paid subscriber you can claim it for free using the discount code in the email header (or ask me for the code).

Get PDF

Number Systems: Decimal, Binary, and Hexadecimal

The Decimal System: Our Familiar Base-10

The decimal system is so natural to us that we rarely think about why we use it. It has 10 distinct digits (0-9), and the position of each digit represents a power of 10:

For example, the following figure shows the representation of the decimal number 4,729

This system is called “base-10” or “radix-10” because it uses 10 as its base value for positional notation.

The Binary System: Base-2

Computers use the binary system (base-2) because digital circuits have two stable states. As we saw in our previous article, transistors function as switches that are either on or off, which naturally maps to 1 and 0. The logic gates built from these transistors process binary data, making binary the native language of all digital hardware.

In binary, each position represents a power of 2 rather than a power of 10:

For example, the following figure shows the broken down representation of the binary number 1011

Which equals 8 + 0 + 2 + 1 = 11 in decimal.

Converting from decimal to binary involves repeatedly dividing by 2 and tracking the remainders. Let’s convert 27 to binary:

27 ÷ 2 = 13 remainder 1 (least significant bit)

13 ÷ 2 = 6 remainder 1

6 ÷ 2 = 3 remainder 0

3 ÷ 2 = 1 remainder 1

1 ÷ 2 = 0 remainder 1 (most significant bit)

Reading the remainders from bottom to top: 11011, which is indeed 27 in binary.

Bit and Byte Terminology

When working with binary data, we need precise terminology to refer to specific portions of the data. The following terms are essential in assembly programming:

Most Significant Bit (MSB) and Least Significant Bit (LSB)

Binary numbers have two "ends" that are particularly important:

Least Significant Bit (LSB): This is the rightmost bit in a binary number. It represents the 2^0 (1) position and contributes the smallest value to the total. The LSB tells us whether the number is odd or even (1 = odd, 0 = even).

Most Significant Bit (MSB): This is the leftmost bit in a binary number. It represents the highest power of 2 in the value and contributes the largest amount to the total.

Most Significant Byte (MSB) and Least Significant Byte (LSB)

When working with multi-byte values (like 16-bit, 32-bit, or 64-bit numbers), we also need terminology for the bytes themselves:

Least Significant Byte (LSB): This is the byte containing the least significant bits of a multi-byte value.

Most Significant Byte (MSB): This is the byte containing the most significant bits.

For example, the 16-bit hexadecimal value 0x4A3F consists of two bytes:

- The MSB is 0x4A

- The LSB is 0x3F

Note about byte order: When multi-byte values are stored in memory, the order of the bytes becomes important. Different computer architectures may store the bytes in different orders (most significant byte first or least significant byte first). This concept, called "endianness," will become relevant when we discuss memory operations in later parts.

The Hexadecimal System: Base-16

Binary representation quickly becomes unwieldy when dealing with larger numbers. A 32-bit value would require 32 binary digits, which is difficult to read and prone to error. This is where the hexadecimal system (base-16) becomes valuable.

Hexadecimal uses 16 symbols: 0-9 and A-F (where A=10, B=11, …, F=15). Each hexadecimal digit represents exactly 4 binary digits (a “nibble”), making conversion between binary and hexadecimal straightforward.

This compact representation makes hexadecimal particularly useful for expressing binary values. For example, the binary number 1011010010011110 can be more compactly written as 0xB49E in hexadecimal. The “0x” prefix is a common notation indicating a hexadecimal number.

To convert this back to decimal:

• B = 11 × 16³ = 11 × 4096 = 45056

• 4 = 4 × 16² = 4 × 256 = 1024

• 9 = 9 × 16¹ = 9 × 16 = 144

• E = 14 × 16⁰ = 14 × 1 = 14

Adding these values: 45056 + 1024 + 144 + 14 = 46238

Why These Number Systems Matter in Assembly

When working with assembly language, you’ll constantly use all three number systems:

1. Binary is the processor’s native language. All the data in memory and registers is represented in binary and understanding this representation makes it easier to manipulate it.

2. Decimal is useful for human-friendly values and calculations.

3. Hexadecimal serves as the standard representation for memory addresses because it is easier to read than binary.

In assembly, you’ll typically express values in decimal or hexadecimal:

10 # Decimal 10

0Ah # Hexadecimal A (decimal 10)

0x0A # Alternative hexadecimal notation

Binary Arithmetic

Having explored how numbers are represented in binary, let’s now look at how computers perform calculations on these binary values.

Binary Addition

Binary addition follows similar rules to decimal addition, but with only two digits:

0 + 0 = 0
0 + 1 = 1
1 + 0 = 1
1 + 1 = 0 with a carry of 1

Let’s add the binary numbers 1011 (11 in decimal) and 101 (5 in decimal):

The result, 10000, is 16 in decimal, which is 11 + 5.

This calculation mirrors exactly what happens in the ALU’s adder circuit we examined previously. The carry bits generated during this process are physically propagated through the full adder circuits chained together to handle multi-bit addition.

The Processor’s Status Flags

To manage the results of operations, processors maintain a set of status flags that indicate various conditions. These flags are stored in a special register called the status register or flags register.

Four particularly important flags are:

1. Carry Flag (CF): Set when an unsigned arithmetic operation produces a carry or borrow

2. Zero Flag (ZF): Set when an operation produces a result of zero

3. Sign Flag (SF): Set when an operation produces a negative result (the most significant bit is 1)

4. Overflow Flag (OF): Set when a signed arithmetic operation produces a result outside the representable range

These flags are automatically updated after most arithmetic and logical operations. They’re crucial for implementing control flow in assembly code, as they allow the program to make decisions based on the results of calculations.

Understanding the Carry Flag

Because of the fixed-width registers in the processor, there is always a likely chance for the arithmetic operations to result in values that are too big to fit in the registers, i.e., the operations result in the generation of a carry.

To track these carries, the processor sets the carry bit in the flags register (in X86-64, the register name is rflags). The carry flag comes handy in several situations. Let’s discuss these.

Detecting Unsigned Overflow

The most straightforward use of the carry flag is to detect when an arithmetic result is too large to fit in the available bits - a condition called overflow.

For example, imagine adding two 8-bit unsigned numbers 242 and 18.

The result, 260, doesn’t fit in 8 bits (which can only represent values from 0 to 255). The “1” at the left falls outside our 8-bit range. The processor sets the carry flag to indicate this overflow condition.

Why is this important? In real programs, if you don’t detect overflow, your calculations will silently produce incorrect results:

• The actual stored result would be just 00000100 (4 in decimal)

• Your program would continue using this wrong value (4) instead of the correct result (260)

Consider an accounting program that adds large financial values, undetected overflow could cause funds to “disappear”!

Multi-Precision Arithmetic

“Multi-precision arithmetic” simply means working with numbers that are larger than what fits in a single register.

For example, let’s say we’re using an 8-bit processor but need to add two 16-bit numbers. We’d need to:

1. Add the lower 8 bits of both numbers

2. Add the upper 8 bits of both numbers

3. Account for any carry from the first addition

Here’s how it works, adding 1000 (0x03E8) + 2000 (0x07D0):

The result is 0x0BB8, which is 3000 in decimal. Without tracking the carry from the first addition, we would get 0x0AB8 (2744), which is wrong.

Assembly languages provide special instructions for these operations. For example, in x86, the adc (add with carry) instruction adds two values plus the carry flag, making multi-precision arithmetic possible.

The Foundation of Comparison Operations

The carry flag is also used for comparing unsigned values. When the processor compares two values, it actually subtracts them and sets flags based on the result, without storing the subtraction result.

For example, when comparing unsigned values A and B:

• If A < B, the subtraction A - B requires a borrow, setting the carry flag

• If A > B, no borrow is needed, clearing the carry flag

• if A == B, the zero flag is set, indicating the values are equal

By checking the value of the carry flag we can figure out the result of the comparison, whether it was true or false, and execute appropriate code. When we learn about implementing conditional flow (if conditions) in assembly, we will see how this comes into action.

Get PDF

Read more

“Do not try to bend the spoon. That's impossible. Instead, only try to realize the truth... there is no spoon.” — The Matrix

Most programmers work comfortably inside layers of abstraction: writing code, calling APIs, using tools, without needing to know what happens underneath. But systems-level thinking is about lifting the hood. It means understanding how things actually work, from source code all the way down to silicon.

This article is where that starts. We’ll build a concrete mental model of how a computer executes instructions, beginning at the hardware level with logic gates and circuits. From there, we’ll step through how those circuits form an ALU, how data moves through registers, and how a CPU follows instructions.

But that’s only part of the story. We’ll also look at how this hardware model shapes everything above it. How compilers turn code into machine instructions. How executables are structured. How the OS lays out a process in memory. None of these make full sense without the layer below.

If you want to understand systems, this is the foundation. You don’t need to memorize how every part works. What matters is building a model that helps you reason through the system when things break or behave in unexpected ways. That’s what systems-level thinking is about.

Let’s get started.

Quick heads-up before you read

This article was originally paywalled as part of my x86 assembly series. If you find it useful, consider upgrading to get access to the full series, and discounts on upcoming courses and books.

Subscribe now

There’s also a PDF ebook version of the series (currently 60 pages across four chapters), available separately. Paid subscribers get 40% off with an annual plan and 20% off with a monthly one — just email me to get your discounted link.

Get EBook

Building a Very Simple Processor

We will learn how the hardware executes code by doing a thought exercise where we construct a very simple processor based on the computational requirements of software, such as a simple calculator.

Even the calculator is capable of doing myriads of computations, so to begin with, we will focus on just one computation: adding two integers. Essentially, we want a simple computer capable of expressing and computing the following computation:

int a = 10
int b = 20
int sum = a + b

To be able to do this, what capabilities do we need in the hardware?

• We need a way to represent information: How do we represent data such as the integers 10 and 20 here, and also how do we tell the hardware that it has to add the two values?

• Storing the input and output data: Where do these values of a and b live?

• Doing the actual computation: How do we perform the addition in the hardware?

This line of inquiry leads us to the work done by Claude Shannon where we will find all our answers.

Encoding Information using Binary

You might be familiar with Claude Shannon’s work on information theory, which is the foundation underlying all modern communication systems and data compression techniques.

As part of this work, he came up with the idea of using electrical switches to encode information as binary data. Essentially, if we represent the state of the circuit when it is closed as 1 and when it is open as 0, then we can encode 1 bit of information in that circuit, and by combining multiple such circuits, we can encode more information.

By encoding information in binary, we can leverage the power of binary arithmetic and Boolean algebra to implement complex computational and logical calculations, giving way for general-purpose computation, which are modern computers.

Transistors as Digital Switches

But modern processors aren’t built using electrical switches, they are built using digital switches that can be turned on and off automatically. This is made possible through transistors.

Transistors are semiconductor devices that act as electronic switches. They conduct current only when the voltage applied to them is above or below a certain threshold, depending on their configuration. This ability to switch on or off based on voltage levels makes them ideal for implementing digital circuits. By precisely controlling the flow of current through circuits built from transistors, digital switches are created. These switches form the fundamental building blocks of all modern chips.

Transistors to Logic Gates

Transistors are the bottommost layer of the computing stack based on which everything else is built. They are combined in specific configurations to build reusable components called logic gates.

You can think of gates as mathematical functions (or, if you prefer code, then a function in code) that takes one or more inputs and produces an output. Because we are working with digital circuits, all the inputs and outputs here are 1s and 0s.

For instance, the NOT gate takes one parameter as input and produces one output. As its name suggests, it inverts its input. So NOT(1) = 0 and NOT(0) = 1.

Similarly, there is an AND gate that takes two inputs and produces one output (you can also make an AND gate that takes a larger number of inputs). Mathematically, it works like this:

AND(0, 0) = 0
AND(0, 1) = 0
AND(1, 0) = 0
AND(1, 1) = 1

We also have an OR gate which works like this:

OR(0, 0) = 0
OR(0, 1) = 1
OR(1, 0) = 1
OR(1, 1) = 1

Finally, there is a very useful gate called XOR:

XOR(0, 0) = 0
XOR(0, 1) = 1
XOR(1, 0) = 1
XOR(1, 1) = 0

Boolean algebra establishes that by combining these basic operations, it is possible to compute any mathematical function, and this is how the computational circuits within the processor are designed. Let’s see how.

Building Computational Circuits from Logic Gates

So we started with the goal of implementing the functionality to add two integers in our simple hardware. And now we know that it can be accomplished using logic gates. The logic circuit which implements binary addition is called an adder. But, before looking at the circuit itself, let’s talk about binary addition.

Again, we can think of it like a mathematical or programming function. It receives two bits as input and produces two bits as output. One of the output bits represents the sum of the two input bits, and the 2nd output bit represents the overflow or carry of the result.

add(0, 0) = sum: 0, carry: 0
add(0, 1) = sum: 1, carry: 0
add(1, 0) = sum: 1, carry: 0
add(1, 1) = sum: 0, carry: 1

What you will see is that the mapping of the input bits to the sum value is identical to that of the XOR gate, while the mapping of the carry bit is identical to that of the AND gate. It means that an adder can be implemented by sending the input to an AND gate and a XOR gate, like the following figure:

This design is called a half adder because it is not useful when adding multibit numbers. For addition of two multibit numbers, we need three inputs: two input bits from the numbers at a given position, and one carry bit from the addition of bits at the previous position. For this, a slightly modified circuit is used, called the full adder.

I will not show its construction because this article is not about digital design, but the following circuit shows what it looks like. By chaining these adders together, we can create circuits capable of adding multibit numbers.

The ALU

A real-world processor consists of multiple different kinds of computational circuits for operations such as addition, subtraction, multiplication, division, and also logical operations (AND, OR, NOT). These circuits are combined in the form of an arithmetic logical unit (ALU) and the various computational circuits within it are called functional units.

The inputs flow into the ALU which activates the right functional unit and produces the output. Schematically for our simple processor it looks the following diagram.

The Need for Storage: Introducing Registers

As you can see in the ALU diagram, it receives some input, performs the computation, and produces an output. So, the question arises: from where do these inputs come and where does the result go afterward? The answer is registers.

Apart from building computational circuits, transistors can be used to construct circuits that can hold state as well, i.e. memory. Using such circuits, we can construct memory units, and one such unit is the register.

Registers are fixed-sized memory units capable of storing a small number of bits, for example, modern processors have 32 or 64 bit wide registers. These are used to temporarily hold the data during computation. For instance, when performing an add operation, the input parameters are first stored in the registers, and then their values are fed into the ALU to perform the computation.

Typically, during program execution, data is moved into the registers from main memory (the RAM) and after the computation is done, the result is written back to the main memory. This frees up the register for other computations.

In our example of building a simple calculator, we need:

• Two registers to hold the numbers we want to add (let’s say R1 and R2)

• One register to hold the result of the addition. However, we could also use one of the two registers to store the output.

But real-world processors have many registers, e.g., the X86 architecture has 16 general-purpose registers. These registers are combined into a register file from which the data flows into the ALU. Let’s update the architecture diagram of our processor to see how it looks after the introduction of a register file consisting of 6 registers :

The diagram has started to show the typical organization of computer hardware at an abstract level. Typically, the ALU isn’t the only execution unit within the processor, so we have abstracted it inside an execution unit. The data flows from the register file into the execution unit to execute the instructions, and an output comes out of the execution units. As we go along, we will furnish more details in the diagram.

At this stage, our ALU is extremely limited; it can only perform an addition. Let’s extend it.

Adding Features and Control to Our Calculator

A processor capable of just addition is not very useful. We need more features, such as subtraction, multiplication, and division. All of these require computational circuits similar to the adder.

For example, subtraction can be implemented using the adder itself by simply negating the second operand value. But operations like multiplication need their own specialized circuits. We can implement these additional operations by adding separate circuits for each one, resulting in a more functional ALU with multiple functional units.

While we discussed the construction of the adder at the level of logic gates, we will not cover the remaining circuits at that level of detail. While that knowledge is valuable, as software engineers, it is not necessary to understand how everything works at the circuit level.

With multiple functional units in the ALU and multiple registers, we need a way to control which operation is performed and which registers are involved. This is done by a component of the processor, called the control unit.

The Control Unit and Decoders

The Control Unit is responsible for managing the flow of data between the registers and the ALU, and for selecting the appropriate functional unit within the ALU. It orchestrates the entire computation process by sending specific control signals to activate exactly the right components at the right time.

To do this, the Control Unit must know three things:

1. What operation needs to be performed. (e.g., addition, multiplication)

2. Where the operand lives. (i.e., which register holds the operand)

3. Where the result needs to be stored. (i.e., the destination register, or the memory address)

This information is provided to the Control Unit in the form of a binary encoded instruction.

Yes, here we are talking about the program that you and I write. Our programs get compiled down to a sequence of binary encoded instructions that the processor can decode and execute. We will talk about program execution later, but right now let’s focus on how the instructions are encoded and decoded.

Instruction Encoding and Decoding

Every hardware architecture usually has its own encoding format, and the details vary. So, instead of focusing on a specific implementation, we can try to get a general understanding of what this whole process involves. Let’s first understand how an encoded instruction may look like.

An instruction in our simple architecture needs to provide three pieces of information:

• The opcode: which operation to perform

• The two operand registers: from where the data for executing the operation comes

• The destination operand register: where does the result go

Now, let’s say our hardware currently supports four operations (+, -, /, *), then we can encode all of it using 2 bits. But to be future proof where we expect more operations to be added, we can make the opcode size as 3 bits and encode these as follows:

000 = Addition  
001 = Multiplication
010 = Subtraction    
011 = Division

Similarly, we have six registers which we can encode using 3 bits:

000 = R1    
001 = R2    
010 = R3    
011 = R4    
100 = R5    
101 = R6

For instance, an instruction to multiply the values in R1 and R2 and store the result in R3 will be encoded as: 001000001010.

To decode the instruction, the control unit contains a decoder - a logic circuit (similar to the adder we saw previously) which maps these opcode and register bits to the right functional unit and registers.

Based on the decoded opcode and register names, the control unit sends control signals to the register file and the ALU which internally use that signal to select the appropriate functional unit (e.g., the adder) and the right register (e.g., R1).

The ALU uses the control signal to switch on the required functional unit. In the case of the register file, a multiplexer is involved which selects the right register in the file, and its data flows into the ALU.

The Instruction Pointer and the Instruction Execution Cycle

Up to this point, we have explained how the Control Unit decodes instructions and orchestrates data movement between the Register File and the ALU. But we haven’t discussed, from where the control unit gets these instructions?

In most hardware architectures, there is a special register called the instruction pointer (often called the program counter). This register’s role is to contain the address of the next instruction of the program.

The control unit reads the address in the Instruction Pointer and fetches the instruction from main memory. The instruction fetched is then stored in another special register called the instruction register (IR). Once the instruction register has a new instruction, the whole process of decoding and executing instruction takes place.

These registers enable what’s known as the instruction execution cycle - the fundamental process by which all programs run:

1. Fetch: The Control Unit reads the address stored in the Instruction Pointer and retrieves the instruction from that memory location, placing it in the Instruction Register.

2. Decode: The Control Unit decodes the instruction in the IR, determining which operation to perform and which registers to use.

3. Execute: The appropriate ALU circuit is activated to perform the computation using data from the specified registers.

4. Store: The result is written back to the destination register (or memory, as needed).

5. Update IP: The Instruction Pointer is incremented to point to the next instruction in memory.

This cycle explains how a program’s instructions are executed one after another. But how is the Instruction Pointer register initialized? The answer is the operating system kernel.

When a program first starts, the operating system loads the program’s instructions into memory and initializes the Instruction Pointer to the address of the first instruction. From that point on, the hardware takes over, executing instructions one by one through this cycle.

But what about Memory?

So far we have covered everything needed to construct a bare-bone hardware capable of doing simple arithmetic. But we have not talked about memory, even though we referred to it a few times. As most of the people reading my blog are experienced programmers, so I expect you already know how memory works. But let’s touch upon it briefly to complete the picture.

The Main Memory

So far, we have only talked about registers as memory. Registers are fast, fixed-sized memory units, but they have two significant limitations:

• Limited Number of Registers:

  • Registers are small and fast, and they are also available in limited numbers. A typical program uses much higher number of variables than the number of registers in the hardware. It necessitates the need for a larger memory which can be used when there are not enough registers.

• Lack of Address-based Access:

  • Registers can only hold primitive data types, e.g. integers and floats. But we need more powerful types, such as arrays and structs to build higher-level programming abstractions. These data types need address-based access.

  • For example, if an array A starts at address 0x04 and each element occupies 4 bytes, then A[2] can be accessed by the address 0x0c. This isn’t possible with registers.

This leads to the requirement of a larger memory with an address-based access scheme, which is what the main memory provides. While we don’t have space to go deeper into the physical construction of main memory, let’s discuss it briefly.

Physical Structure of Main Memory

Physically, the main memory is implemented as an array of memory cells, each storing one bit of information. These cells are arranged on silicon chips in a grid of rows and columns for manufacturing reasons. However, the memory is presented to the programmer as a linear sequence of bytes, each with a unique address. There are two types of main memories depending on their physical construction:

• Dynamic RAM (DRAM): The most common type of main memory. Each cell is composed of a capacitor and a transistor. The capacitor holds a charge to represent a 1 or 0, but the charge leaks over time, so the cells must be periodically refreshed. This makes DRAM relatively slower compared to registers.

• Static RAM (SRAM): Used primarily for CPU caches. Each cell is made from flip-flops instead of capacitors, making it faster and more reliable but also more expensive and power-hungry.

Communication Between CPU and Memory

Interacting with the memory is not straightforward. There are multiple pieces of information required to access it. Such as

• Are we reading from memory or writing to it?

• What is the address where the operation is to be performed?

To solve this communication challenge, there are multiple buses between the processor and the memory. These are

• Control Bus: Specifies whether the operation is read or write.

• Address Bus: Carries the memory address specifying where to read from or write to.

• Data Bus: Transfers the actual data between the processor and the memory.

For instance, when the control unit needs to fetch an instruction, here are the things it needs to do:

• Read the instruction address from Instruction Pointer and place it in the address bus

• Update the control bus to indicate the read operation

• Read the returned instruction from the data bus

Apart from the processor using memory to fetch program instructions, program instructions can also read or write memory. Let us discuss how that is made possible.

The Load/Store Units

We already discussed that the control unit fetches the instructions from main memory and puts them in the instruction register for decoding and execution.

Apart from that, the program instructions can themselves read or write memory. For instance, a program that iterates through an array of integers to sum them up has to bring each element of the array from memory into one of the registers to update the sum (which itself will be kept in a register).

Most hardware architectures have instructions that can move data between registers and memory, in both directions. For example, the mov instruction in x86 can be used to move data between the registers, and also between a register and the memory. At the hardware level, the CPU contains special execution units called the Load and Store units that are activated to execute such instructions.

The following diagram shows the processor architecture after the introduction of memory.

There is No Spoon

This is a great place to mention an interesting observation that you will appreciate.

As far as the processor is concerned, everything is just bits. The hardware doesn't see integers, floating-point numbers, strings, or even code—it only processes binary data. The distinction between these abstractions exists only in our minds or at higher layers of abstraction, such as programming languages and compilers.

The hardware doesn't know whether a value in memory is float or an int. It merely moves bits around according to the instructions it's given. For example, most processors have separate sets of registers for integer and floating-point values. When you instruct the processor to move data from memory to a register, it performs that operation without ever verifying the nature of the data.

If you mistakenly load a 32-bit floating-point value into an integer register, the hardware will interpret those bits as an integer. It doesn't validate, interpret, or question, it only executes. The compiler enforces type safety in higher-level languages, but when you're working with assembly, you are responsible for ensuring that the right instructions operate on the right data. There’s no safety net.

Even the distinction between code and data is just another layer of abstraction. When the Instruction Pointer fetches an instruction from memory, the processor simply decodes the bits as an instruction. If you somehow update the Instruction Pointer with the address of a piece of data, such as an integer, the control unit will still read it and attempt to decode it as an instruction. If the bit pattern is invalid as an instruction, the hardware will raise an exception and the program will crash. But it doesn’t do this because it realizes you provided data instead of code, it only fails because the bits didn’t correspond to a valid instruction.

In the end, the processor does not recognize or care about your abstractions. It only sees bits. This realization is similar to the moment in The Matrix when the child tells Neo: “There is no spoon.” .

From High-level code to the Hardware-level Execution

We have covered the fundamentals of how the hardware is organized, now let’s connect everything from top-to-bottom and see how a high-level program is translated to machine code and executed.

Compilation of High-Level Code to Machine Code

As we have seen, the hardware understands only instructions encoded in binary form. But, writing programs directly in binary is cumbersome and error-prone, which is why hardware designers define mnemonic instructions: cryptic but human-readable representations of these binary instructions. This is known as the assembly language.

For example, in x86 assembly, the following instruction adds the values in the registers rax and rdx, and stores the result back into rdx.

add %rax, %rdx

rax and rdx are register names in x86-64 architecture, and the assembler syntax requires using % to represent register names. Don’t worry about these details, we will cover them during the course.

Even writing assembly by hand is cumbersome, so we use high-level languages which are compiled down to assembly or directly to machine code for execution. Consider the following high-level C code:

long a = 10;
long b = 20;
long sum = a + b;

The C compiler will compile this into assembly code which may look like this:

movq $10, %rax ; Store 10 in register rax
movq $20, %rdx ; Store 20 in register rdx
addq %rax, %rdx ; rdx = rax + rdx

The ; character is used to write comments in some assemblers.

Here’s what is happening in the code:

• The movq instructions are used in x64 assembly to write a long (64 bit integer) value into a register. So, the two movq instructions are writing the values 10 and 20 into rax and rdx, respectively.

• The addq instruction is used to add the values of the two registers. The 2nd register is used as the destination. So, this equivalent to rdx = rax + rdx.

But, this assembly code isn’t what the hardware understands. So, there is another translation step where this assembly code is converted into machine code by a tool called the assembler.

The assembler translates one assembly source file at a time and creates a binary file containing the machine code. This file is called the object file. If there are multiple assembly source files, then the assembler creates one object file for each of them.

Finally, to produce the executable file, a third tool, called the linker is used that stitches these object files together into a single file. The role of the linker is to resolve the addresses of the symbols and functions across multiple files.

For instance, if your C program consists of two source files: main.c and math.c, where main.c calls functions defined in math.c, then the compiler and assembler will produce two object files: main.o and math.o. Afterwards, the linker will generate the final executable such that it contains the code for all the functions called from main.

The output of the linker is the final executable binary file. Even though it is a binary file, its format has to be understandable by the operating system because ultimately it is the OS which has to load it into memory for execution.

The Executable Binary Format

To execute a program, the operating system needs to load it into memory and update the Instruction Pointer (IP) register with the address of the first instruction. For this, the OS must understand the format of the executable binary file.

Operating systems standardize the format of binaries. For example, Linux and BSD systems use the Executable and Linkable Format (ELF).

In ELF, the binary file is organized as a sequence of sections, where each section stores a particular kind of data. For example:

• The text section holds the binary encoded instructions (the machine code).

• The data section holds the statically declared program data (e.g., your globally initialized variables).

• The bss section holds uninitialized data.

Note: There are many more kinds of sections and the overall ELF format is more detailed, but we don’t need to know all that right now. Knowledge of the sections and why they exist will be useful when reading/writing assembly code.

The organization into sections serves two purposes:

• Contiguous Instruction Storage: The program instructions need to be stored contiguously in memory so that the hardware can simply increment the Instruction Pointer (IP) by the size of the instruction to get the next instruction address.

• Permission Management: Different kinds of data require different permissions. For example:

  • Program code should be readable and executable, but not writable.

  • Data may be readable and writable, but not executable.

  • By segregating different kinds of data, the OS can set appropriate permissions to enhance security and stability.

This well defined format enables the operating system to parse the file and load the different sections in different regions in memory with appropriate permission flags.

Loading the Program into Memory

When we execute a program, the operating system loads all the code and data into memory before execution begins. Because the operating system understands the binary format, this process is straightforward. The OS allocates memory pages in different regions of memory for different sections of the binary:

• The text section is placed in executable memory.

• The data section is placed in read/write memory.

After setting up the pages and loading all the data, the operating system updates the Instruction Pointer with the address of the first instruction of the program. This is possible because the OS knows exactly where the code was loaded in memory.

Once this setup is done, control is transferred to the hardware to begin executing the program.

The Instruction Fetch-Decode-Execute Cycle

Now that the program is loaded into memory, the Control Unit takes over and begins executing instructions by repeating the Fetch-Decode-Execute Cycle:

• Fetch: The Control Unit fetches the next instruction from memory using the address stored in the Instruction Pointer (IP).

• Decode: The Control Unit decodes the instruction to determine the operation, source operands, and destination register.

• Execute: The ALU performs the specified operation.

• Store: The result is written back to the appropriate register or memory location.

• Increment IP: The Instruction Pointer (IP) is incremented to point to the next instruction, and the cycle repeats until the program ends.

This cycle is fundamental to how all processors operate. Each instruction is processed through these stages in a continuous loop until the program completes.

Conclusion

We began this article by asking a simple question: What does it take to build a processor for a simple computer that can add two numbers? From this modest starting point, we’ve uncovered the fundamental architecture that underpins all modern computing.

Our simple processor led us to discover key components that exist in every computer:

• Transistors and Logic Gates: The basic building blocks that implement Boolean operations and enable computation at the physical level.

• ALU: The computational core that began as a simple adder in our processor but expands to handle diverse operations in real processors.

• Registers: Fast, accessible storage locations that hold the data being actively processed.

• Control Unit: The orchestrator that decodes instructions and coordinates all components, determining which operations to perform and on what data.

• Instruction Execution Cycle: The fundamental fetch-decode-execute cycle that drives all program execution.

• Memory System: The larger storage hierarchy that holds both instructions and data beyond what fits in registers.

While our simple processor serves as a clean conceptual model, real architectures like x86 and ARM are much more sophisticated. They incorporate advanced features such as pipelining, branch prediction, cache hierarchy.

Yet despite this complexity, these advanced processors still follow the same fundamental organization and principles we’ve explored. They still fetch instructions from memory, decode them to determine operations, execute using an ALU, store results, and move to the next instruction.

By understanding this bottom layer, how hardware actually implements computation, you now have a solid foundation for learning assembly programming.

In our upcoming x86 assembly course, we’ll build on this foundation by exploring the specific registers, instructions, and memory addressing modes of the x86 architecture. We’ll connect high-level programming constructs to their low-level implementation, allowing you to see through the abstractions to the true nature of computation - just as Neo finally saw the Matrix for what it really was.

Share

What to Read Next

Real-world processors have many more advanced features for delivering high performance, such as instruction pipelining, caches, and branch prediction. If you are curious, then my recent article is the perfect stepping stone to learn about them.