How Computer Memory Works — From CPU Registers to Disk Swap

Since I’ve started learning how computers actually work under the hood, one of the topics that completely changed how I see system performance is something called the Memory Hierarchy.

It may sound abstract at first, but this hierarchy determines how fast, how reliable, and how efficient your system is when dealing with data — whether it’s opening an app, loading a website, or compiling code.

In this post, I’ll walk you through the different layers of computer memory including RAM, ROM, Cache, Virtual Memory, Swap space, and Page Tables. And I’ll try to make it all make sense using real-world examples and analogies.

What Is Memory Hierarchy?

At its core, the Memory Hierarchy is a way to organize different types of computer memory based on:

• Speed
• Size
• Cost
• Proximity to the CPU

It looks something like this:

CPU Registers
    ↓
L1 Cache (fastest, smallest)
    ↓
L2 / L3 Cache
    ↓
Main Memory (RAM)
    ↓
Storage (HDD / SSD)
    ↓
Swap Space / Virtual Memory

Each level trades speed for capacity. Faster memory is smaller and more expensive. Slower memory is larger and cheaper.

1. CPU Registers

• What are they? Smallest and fastest memory units inside the CPU itself.
• Size: A few bytes per core (think 32-bit or 64-bit values).
• Latency: Almost zero — directly wired to the CPU.
• Use case: Store immediate data for arithmetic or logic operations.

Analogy: Like a calculator’s memory — very quick but can only store a tiny number at a time.

2. Cache Memory (L1, L2, L3)

Cache is small memory located very close to (or inside) the CPU. It stores data that the processor is likely to reuse soon.

L1 Cache:

• Fastest but very small (32KB – 128KB).
• Split into instruction cache and data cache.

L2 Cache:

• Larger (256KB – 1MB), a bit slower.
• Usually per core.

L3 Cache:

• Shared between CPU cores.
• Larger (4MB – 64MB), slower than L2 but still much faster than RAM.

Why Cache Matters:
Modern CPUs can execute billions of instructions per second, but accessing RAM takes hundreds of cycles. Cache bridges that gap.

Real Example:
You’re scrolling through a document. CPU cache stores the next few lines in advance, so scrolling feels instant.

3. Main Memory: RAM (Random Access Memory)

RAM is what we typically call “memory” when we say “This laptop has 16GB of RAM.”

Characteristics:

• Volatile memory: Data is lost when power is off.
• Faster than storage, but slower than cache.
• Holds active programs and data that CPU can access relatively quickly.

Use Case:

• Operating System
• Open applications
• Loaded files

Analogy: Think of RAM as your desk. You put the books (programs) you’re currently reading on it. But there’s limited space.

4. ROM (Read-Only Memory)

ROM is non-volatile memory — contents persist even when power is off.

• BIOS/UEFI firmware
• Embedded systems (e.g., routers, microcontrollers)

Once written (or “flashed”), ROM can’t be easily modified — it stores critical low-level code like how to boot your system.

5. Virtual Memory

Virtual Memory is a software abstraction that allows systems to treat storage (like SSD/HDD) as if it were additional RAM.

• Managed by the Operating System
• Uses both RAM and disk
• Divides memory into pages

Even if your RAM is full, the system can keep running by temporarily moving inactive data to the disk.

Analogy: You have too many books for your desk (RAM), so you store some under the table (Virtual Memory) and grab them when needed.

6. Swap Space

Swap is the actual area on disk used for virtual memory.

• On Linux: Swap partition or swapfile.
• On Windows: Pagefile.

When RAM is full, OS moves less-used memory pages to swap.

Downside: Disk is much slower than RAM → can cause system slowdowns if used excessively (called “thrashing”).

Real Example: Ever opened Chrome with 100 tabs on an 8GB RAM laptop? That lag is your OS desperately using swap.

7. Page Tables

• Data structures managed by the Memory Management Unit (MMU)
• Translate virtual addresses to physical memory addresses

Each process thinks it has its own memory space. Page tables make that illusion possible.

Without page tables, programs would conflict over memory and crash each other constantly.

Fun Fact: Most systems use multi-level page tables (e.g., 4 levels in x86-64 architecture) to manage large address spaces efficiently.

Real-World Scenario: What Happens When You Open an App?

Let’s say you double-click on Visual Studio Code. Here’s what happens behind the scenes:

1. User Clicks the Icon
   The OS receives the click event.
2. The OS Locates the Executable
   It finds the binary on disk — it’s not in memory yet.
3. Loading Into RAM
   The OS copies the program and its dependencies from disk into RAM. Memory pages are allocated.
4. Virtual Memory Mapping Begins
   The MMU uses page tables to map the program’s virtual memory space to physical RAM.
5. The CPU Starts Execution
   The CPU fetches instructions. It first checks L1 cache, then L2, then L3. If not found, it reads from RAM.
6. The Program Opens Files
   VS Code loads your previous project files. The OS reads them from disk into RAM.
7. RAM Gets Full – Swap Activates
   If RAM usage is high, the OS moves less-used pages to swap space on disk.
8. Continuous Cycle
   As you use the app, the CPU, cache, RAM, and disk work together in a constant loop to serve you data.

Real-World Relevance

Layer	Speed	Example Use Case
Registers	Ultra-fast	Add 2 numbers
Cache (L1–L3)	Fast	Reuse loop variables, function data
RAM	Moderate	Active programs, tabs, documents
Virtual Mem	Slow	Backgrounded apps, large datasets
Disk (Swap)	Very Slow	Emergency memory fallback

The Memory Hierarchy is a brilliant compromise between speed, size, and cost. It allows computers to:

• Be fast when they need to be
• Store large amounts of data when necessary
• Keep multiple applications running simultaneously
• Protect memory spaces between applications