Back to series
· 8 min read

How does a CPU work? Understanding the processor from instruction execution


I. What Is a CPU?

A CPU (Central Processing Unit) is the core computing and control component of a computer. It executes program instructions, processes data, and coordinates the other hardware components to get work done.

Fundamentally, a CPU is a digital circuit system that continuously fetches instructions, decodes them, executes operations, and updates its processing state according to the rules defined by its instruction set architecture (ISA).

Whether running an operating system, compiling a program, or executing a database query or an AI inference, everything must ultimately be converted into machine instructions that the CPU can recognize and execute, or be handled cooperatively by other specialized processors.

1.1 Core Responsibilities of the CPU

The CPU mainly handles the following work:

Responsibility Description
Instruction control Fetches, decodes, and controls instruction execution
Arithmetic operations Performs addition, subtraction, multiplication, division, and other arithmetic operations
Logic operations Performs comparison, AND, OR, NOT, and other logical operations
Data processing Reads, transforms, and writes data
Flow control Performs jumps, branches, and similar operations based on conditions
System coordination Works with memory, I/O devices, and other hardware

1.2 Where the CPU Sits in a Computer

The CPU does not work on its own; together with memory, storage devices, and input/output devices, it forms a complete computer system.

flowchart TB
    CPU["CPU<br/>Instruction execution and control"]
    MEM["Memory<br/>Programs and runtime data"]
    IO["I/O devices<br/>Keyboard, monitor, network card"]
    DISK["Storage devices<br/>SSD, HDD"]

    CPU <--> MEM
    CPU <--> IO
    MEM <--> DISK

    classDef cpu fill:#DCEFE3,stroke:#6DAD86,color:#285C40,stroke-width:1.5px
    classDef mem fill:#DFEAF7,stroke:#82A8D1,color:#315579,stroke-width:1.5px
    classDef io fill:#F8EEDB,stroke:#D4B77C,color:#735B2D,stroke-width:1.5px
    classDef disk fill:#EAE5F4,stroke:#A99BC8,color:#574A78,stroke-width:1.5px

    class CPU cpu
    class MEM mem
    class IO io
    class DISK disk

The CPU executes instructions, memory provides the data and instructions the program needs to run, storage devices keep data for the long term, and input/output devices handle interaction with the external environment.

Note that in modern computers, the CPU, memory, and I/O devices are not simply connected directly to one another; they work together through components such as memory controllers, buses, and I/O controllers.

II. The Internal Composition of a CPU

Internally, a CPU is made up of multiple functional units. Each unit has its own role, and together they carry out instruction fetch, decode, execution, and result commit.

From a functional perspective, the core components of a CPU can be divided into:

  • Control Unit (CU)
  • Arithmetic Logic Unit (ALU)
  • Register file (Registers)
  • Cache
  • Clock and related control logic

2.1 CPU Core Structure

flowchart TB
    subgraph CPU["CPU processor"]
        direction TB

        CU["Control Unit (CU)<br/>Instruction decoding and control"]
        REG["Registers<br/>Temporarily hold instructions and data"]
        ALU["Arithmetic Logic Unit (ALU)<br/>Arithmetic and logic operations"]
        CACHE["Cache<br/>Reduces memory access latency"]
        CTRL["Clock and control logic<br/>Coordinates the execution process"]

        CU <--> REG
        REG <--> ALU
        CU <--> ALU
        CACHE <--> REG
        CTRL --> CU
        CTRL --> ALU
    end

    MEM["Main memory (RAM)"]

    CACHE <--> MEM

    classDef cu fill:#DCEFE3,stroke:#6DAD86,color:#285C40
    classDef reg fill:#DFEAF7,stroke:#82A8D1,color:#315579
    classDef alu fill:#F8EEDB,stroke:#D4B77C,color:#735B2D
    classDef cache fill:#EAE5F4,stroke:#A99BC8,color:#574A78
    classDef ctrl fill:#E5ECE9,stroke:#91AAA0,color:#3C5C50
    classDef mem fill:#DFEAF7,stroke:#82A8D1,color:#315579

    class CU cu
    class REG reg
    class ALU alu
    class CACHE cache
    class CTRL ctrl
    class MEM mem

The diagram above is an abstract structure at the functional level; it does not represent how every CPU is actually wired internally. Modern processors also contain complex structures such as fetch units, execution ports, load/store units, and reorder buffers.

2.2 Control Unit (CU)

The Control Unit is responsible for coordinating the work of the various components inside the CPU.

Main responsibilities:

  • Fetch instructions from the instruction stream.
  • Decode machine instructions.
  • Identify the operands required by an instruction and the type of execution.
  • Generate the corresponding control signals.
  • Coordinate the flow of data among registers, the arithmetic unit, and memory.

For example, when executing an addition instruction, the control unit needs to recognize that this is an arithmetic operation and coordinate reading the operands, performing the addition, and saving the result.

The control unit itself is not responsible for performing the actual arithmetic operation.

2.3 Arithmetic Logic Unit (ALU)

The ALU (Arithmetic Logic Unit) is the core component of the CPU that performs arithmetic and logical operations.

Main operations include:

Operation Type Example
Arithmetic operations Addition, subtraction
Logical operations AND, OR, XOR, NOT
Comparison operations Greater than, less than, equal to
Bit operations Left shift, right shift

For example:

ADD R1, R2, R3

This means adding the values in registers R2 and R3 and writing the result to R1.

R1 = R2 + R3

Here, the ALU is responsible for performing the addition operation, while the control logic is responsible for coordinating the entire execution process.

Note that multiplication, division, and floating-point operations are usually performed by dedicated execution units in modern CPUs and are not necessarily all handled by the ordinary integer ALU.

2.4 Registers

Registers are extremely fast, small-capacity storage units inside the CPU, used to temporarily hold instructions, data, addresses, and execution state.

Common registers:

Register English Main Function
Program Counter PC Holds the address of the next instruction to be fetched
Instruction Register IR Holds the current instruction (conceptual model)
General-Purpose Register GPR Stores operands and results of operations
Stack Pointer SP Points to the current stack location
Status Register Flags Holds condition codes and status information

Among these, the program counter (PC) is especially important.

The CPU uses the PC to determine the address of the next instruction to be fetched. When execution proceeds in normal order, the PC is updated to the address of the next instruction; when a jump instruction is encountered, the PC may be changed to the target address.

Modern CPUs may track multiple in-flight instructions simultaneously, so the program counter and related state actually used internally are more complex than in the simple model.

2.5 Cache

The CPU's computation speed is much higher than the access speed of main memory.

If the CPU had to access main memory every time it reads data, it could wait a long time for the data to return, preventing compute resources from being fully utilized.

Cache is a high-speed cache located between the CPU and main memory, used to hold data and instructions that were recently used or are likely to be used again.

Common three levels of cache:

Type Location Characteristics
L1 Cache Inside the CPU core Small capacity, fastest, usually split into instruction cache and data cache
L2 Cache Inside or near the CPU core Larger capacity, slower access than L1
L3 Cache More commonly shared among multiple cores Even larger capacity, access speed usually lower than L2

Typical data access hierarchy:

flowchart TB
    CPU["CPU Execution Unit"]
    L1["L1 Cache<br/>Fastest, smallest capacity"]
    L2["L2 Cache<br/>Faster"]
    L3["L3 Cache<br/>Larger capacity"]
    RAM["Main Memory RAM<br/>Large capacity, high latency"]
    SSD["SSD / HDD<br/>Long-term storage"]

    CPU --> L1
    L1 --> L2
    L2 --> L3
    L3 --> RAM
    RAM --> SSD

    classDef cpu fill:#DCEFE3,stroke:#6DAD86,color:#285C40
    classDef cache fill:#DFEAF7,stroke:#82A8D1,color:#315579
    classDef memory fill:#F8EEDB,stroke:#D4B77C,color:#735B2D
    classDef storage fill:#EAE5F4,stroke:#A99BC8,color:#574A78

    class CPU cpu
    class L1,L2,L3 cache
    class RAM memory
    class SSD storage

When the CPU needs to read data, it first checks the nearer cache levels.

  • Cache Hit: The data is already present in the cache and can be read directly.
  • Cache Miss: The target data is not in the cache and must be fetched from a lower level.

The higher the cache hit rate, the less time the CPU typically spends waiting for memory data.

The design of cache mainly relies on two important properties:

Temporal Locality

Data that has been accessed recently may be accessed again in the future.

For example, a variable in a loop may be used repeatedly within a short period of time.

Spatial Locality

After a memory address is accessed, nearby addresses may be accessed soon afterward.

For example, sequential traversal of an array.

This is also why sequentially accessing an array usually delivers better performance than randomly accessing memory.

3. How Does the CPU Execute an Instruction?

To understand how the CPU works, the most important thing is to understand the instruction execution cycle.

From the classic principles of computer organization, the execution of an instruction can be abstracted into four stages:

  1. Fetch
  2. Decode
  3. Execute
  4. Write Back

These four stages form the classic instruction execution model.

3.1 Instruction Fetch

The CPU first needs to know where the next instruction is.

The program counter (PC) holds the address of the next instruction to be fetched.

The fetch process:

  1. The CPU obtains the instruction address from the PC.
  2. It requests the data at that address from the instruction cache or memory.
  3. It fetches the machine instruction.
  4. It sends the instruction to the subsequent processing stages.
  5. It updates the PC so that it points to the address of the subsequent instruction (the exact update method depends on the instruction and the processor implementation).

Illustration:

PC = 0x1000

Read the instruction at memory address 0x1000

Fetch instruction:
ADD R1, R2, R3

PC updated to the address of the next instruction

The address 0x1000 here is only an example, indicating the instruction's location in the memory address space.

3.2 Decode

After fetching a machine instruction, the CPU needs to understand what the instruction means.

The decode stage mainly identifies:

  • Opcode: what operation needs to be performed.
  • Operand: the data involved in the operation.
  • Addressing mode: how to obtain the operands.
  • Execution type: which execution unit needs to handle it.

For example:

ADD R1, R2, R3

After decoding, the CPU can determine:

Operation type: addition
Source operands: R2, R3
Destination: R1
Execution unit: integer arithmetic unit

Note: the above is an assembly representation for ease of understanding; actual machine instructions are made up of binary encodings, and different ISAs use different encoding formats.

3.3 Execute

After decoding is complete, the CPU hands the instruction to the corresponding execution unit.

For example:

R2 = 10
R3 = 20

ADD R1, R2, R3

The ALU performs:

10 + 20 = 30

The calculation result is temporarily held in the datapath or relevant registers during execution, waiting for the subsequent write-back.

Different types of instructions invoke different execution units:

Instruction Type Execution Unit
Integer addition/subtraction Integer ALU
Integer multiplication/division Integer multiply/divide unit
Floating-point operations Floating-point execution unit
Data load Load/store unit
Conditional branch Branch execution unit

3.4 Write Back

After the operation is complete, the result needs to be written to the destination.

For example:

R2 = 10
R3 = 20

ADD R1, R2, R3

The final result:

R1 = 30

At this point, execution of this instruction is complete.

However, not all instructions need to write back to a register. For example, some store instructions write data to memory, while branch instructions may update the program execution location.

3.5 Complete Instruction Execution Flow

flowchart TB
    A["Fetch<br/>Get instruction according to PC"]
    B["Decode<br/>Parse opcode and operands"]
    C["Execute<br/>Invoke the corresponding execution unit"]
    D["Write Back<br/>Save execution result"]
    E["Next instruction"]

    A --> B
    B --> C
    C --> D
    D --> E
    E --> A

    classDef fetch fill:#DCEFE3,stroke:#6DAD86,color:#285C40
    classDef decode fill:#DFEAF7,stroke:#82A8D1,color:#315579
    classDef execute fill:#F8EEDB,stroke:#D4B77C,color:#735B2D
    classDef write fill:#EAE5F4,stroke:#A99BC8,color:#574A78

    class A fetch
    class B decode
    class C execute
    class D write
    class E fetch

This is the classic instruction execution cycle.

But it must be made clear: this is a simplified model for understanding how the CPU works. Modern high-performance CPUs do not strictly wait for one instruction to finish before starting the next; instead, through techniques such as pipelining and out-of-order execution, they keep multiple instructions in different processing stages at the same time.

IV. How Does the CPU Improve Instruction Execution Efficiency?

If the CPU had to fully execute one instruction before it could begin processing the next, a large amount of hardware resources might go underutilized.

To improve processing efficiency, modern CPUs introduce a variety of optimization techniques.

4.1 Instruction Pipeline

Pipelining is an important technique by which modern processors increase instruction throughput.

Its core idea is:

Split the instruction execution process into multiple stages, so that multiple instructions are in different stages at the same time.

You can think of it as an assembly line:

  • The first instruction is executing.
  • The second instruction is being decoded.
  • The third instruction is being fetched.

Different instructions can occupy different processing stages in parallel.

Suppose an instruction needs to go through five stages:

Stage Name Main Work
IF Instruction Fetch Fetch instruction
ID Instruction Decode Decode
EX Execute Execute
MEM Memory Access Memory access
WB Write Back Write back

Pipeline execution illustration:

flowchart TB
    subgraph S1["Clock cycle 1"]
        direction LR
        A1["Instruction 1: IF"]
    end

    subgraph S2["Clock cycle 2"]
        direction LR
        A2["Instruction 1: ID"]
        B2["Instruction 2: IF"]
    end

    subgraph S3["Clock cycle 3"]
        direction LR
        A3["Instruction 1: EX"]
        B3["Instruction 2: ID"]
        C3["Instruction 3: IF"]
    end

    subgraph S4["Clock cycle 4"]
        direction LR
        A4["Instruction 1: MEM"]
        B4["Instruction 2: EX"]
        C4["Instruction 3: ID"]
        D4["Instruction 4: IF"]
    end

    subgraph S5["Clock cycle 5"]
        direction LR
        A5["Instruction 1: WB"]
        B5["Instruction 2: MEM"]
        C5["Instruction 3: EX"]
        D5["Instruction 4: ID"]
        E5["Instruction 5: IF"]
    end

    S1 --> S2 --> S3 --> S4 --> S5

    classDef stage fill:#DCEFE3,stroke:#6DAD86,color:#285C40,stroke-width:1.2px

    class A1,A2,A3,A4,A5,B2,B3,B4,B5,C3,C4,C5,D4,D5,E5 stage

    linkStyle 0,1,2,3 stroke:#82A8D1,stroke-width:2px