Pipeline Speedup: What Actually Improves Throughput in Pipelined CPUs

Pipeline throughput is limited by how often the pipeline can advance each cycle without being stopped. Classic idealized “pipeline speedup” assumes no pipeline hazards and perfect overlap of work. In practice, speedup with additional hardware resources is constrained by (1) control/data hazards, (2) structural hazards, and (3) the memory hierarchy (cache/RAM latency and bandwidth).

To connect your multiple-choice claim to architecture reality, treat “speedup” as the improvement in cycles per instruction (CPI) or instructions per cycle (IPC) as the design evolves.

Key terms:

• pipeline
• CPI
• hazard
• structural hazard
• ideal speedup

The “ideal pipeline speedup” (textbook baseline)

In the ideal model with $s$ pipeline stages, the pipeline can fill in $s-1$ cycles, then sustain one completed instruction per cycle. If there are $n$ instructions, a common idealization yields:

$$\text{Speedup} \approx \frac{n \cdot s}{n + (s-1)}$$

For large $n$, the speedup approaches about $s$ (up to the assumptions of the model). This “speedup increases with number of pipeline stages” is true only in the ideal hazard-free setting. (As stage count grows in real designs, clock rate and forwarding complexity often make the ideal assumptions less valid.)

Key terms:

• pipeline depth
• pipeline fill time
• throughput

Evaluating the four options: (i) control units, (ii) RAM size, (iii) pipeline stages, (iv) cache size

(iii) Number of pipeline stages — Yes, in the ideal model

In the ideal pipelining model, adding stages increases the maximum overlap, so the ideal speedup trend is increasing with pipeline stage count $s$. The fill latency becomes larger ($s-1$ cycles), but for long instruction streams the steady-state dominates.

However, real pipelines often trade stage count against:

• clock frequency
• forwarding
• hazard detection

So: (iii) is correct only under ideal assumptions.

Mermaid intuition:

(i) Number of control units — usually No

Control units (or control logic variants) matter mainly when the design is superscalar/out-of-order/multi-issue, where multiple instructions are decoded and directed concurrently. In a simple single-issue pipeline, adding “more control units” does not directly reduce hazards caused by true dependencies, nor does it change the core pipeline overlap model $s$.

So the generic “ideal pipeline speedup increases with number of control units” is not supported by the classic pipelining speedup model. In superscalar designs, the relevant quantity is issue width and the ability to execute multiple independent instructions per cycle—not simply the count of “control units.”

Key terms:

• issue width
• superscalar
• decode stage

(ii) Size of RAM — No

RAM capacity does not reduce the latency per memory access in the way needed for “ideal pipeline speedup.” Even with larger RAM:

• the pipeline still stalls when the CPU must wait for main memory data;
• cache hit/miss behavior is the dominant factor for stall frequency.

RAM size can affect working set fit at a high level (bigger memory can prevent paging or allow larger resident datasets), but “pipeline speedup increases with RAM size” is not a statement about pipelining itself; it is about whether the machine avoids memory pressure and swapping.

Key terms:

• working set
• main memory latency
• stall cycles

(iv) Size of cache — Yes (practically), but not in the ideal stage-count model

In real CPUs, increasing cache size can reduce the miss rate and therefore reduce average memory stall time, improving effective CPI and increasing IPC. This yields “speedup” in practice because stalls are reduced.

However, the relationship is not monotonic forever (caches have diminishing returns and cost/latency tradeoffs). Still, compared to the ideal pipeline formula, cache size directly impacts how often hazards translate into stalls, especially load/store.

Key terms:

• cache hit
• cache miss
• miss rate
• AMAT