Instruction Pipelining: Meaning, Rationale, Working Procedure, and Performance Measures

Instruction pipelining is a hardware mechanism in which the execution of instructions is split into multiple stages, so that different instructions progress through different stages at the same time. The motivation is to improve processor throughput, by increasing the rate at which instructions finish, rather than only reducing the time for a single instruction.

Conceptually, in a non-pipelined design, instructions run to completion before the next begins, leading to idle time in different functional units. In an n-stage pipeline, once the pipeline is filled, many instructions are “in flight” simultaneously, one in each stage—subject to pipeline hazards.

A typical textbook rationale: pipelining reduces the effective latency per instruction at the system level by allowing overlap, while the cycle time is reduced to match the slowest stage delay. After fill, the ideal steady-state completion rate is one instruction per cycle (for certain hazard-free pipelines). This is commonly expressed in terms of ideal CPI and throughput.

Note: Real pipelines may require stalls due to hazards, which reduce ideal performance.

Key terms (for the rest of the section)

• Pipeline stage
• Pipeline fill
• Pipeline bubble
• Pipeline flush

These terms will be used when explaining working procedure and performance.

Pipeline timing diagram (hazard-free ideal)

Below is a generic 5-stage example (IF, ID, EX, MEM, WB). Each instruction advances one stage per cycle.

In the ideal, hazard-free case, after fill the machine completes one instruction per cycle (steady state). The first completion occurs after approximately $n$ cycles due to pipeline fill.

Pipeline Performance Measures

Performance in pipelined processors is typically evaluated using metrics that relate to:

• clock cycle time ($T_{clk}$),
• latency to finish an instruction,
• throughput (instructions per unit time),
• CPI (cycles per instruction),
• and speedup relative to a baseline.

1) Latency vs throughput

• Latency: time from when an instruction starts to when it completes. In an $n$-stage pipeline, ideal latency is about $n$ cycles (plus any additional bubbles due to hazards).
• Throughput: completion rate after the pipeline is filled. In the ideal hazard-free steady state, throughput is $1$ instruction/cycle.

2) CPI and effective CPI

CPI is computed using the effective number of cycles required for a workload.

• If an $n$-stage pipeline executes a program of $m$ instructions with no stalls, the total cycles are approximately: $\text{Cycles} \approx (n-1) + m$ so the average CPI is: $\text{CPI} \approx 1 + \frac{n-1}{m}$ As $m$ becomes large, CPI approaches $1$.

• With hazards, stalls insert extra cycles. A common qualitative view is: $\text{CPI} = 1 + \text{(average stall/bubble contribution)}$ where the stall contribution depends on hazard frequency and stall duration.

3) Throughput

Throughput (instructions per second) depends on CPI and clock frequency $f_{clk}$: $\text{Throughput} = \frac{f_{clk}}{\text{CPI}}$

This emphasizes why pipelining is valuable: even if CPI slightly exceeds $1$, throughput can increase substantially compared to a non-pipelined baseline.

4) Speedup

A common baseline is a non-pipelined machine where each instruction takes $k$ cycles. If the pipelined machine’s effective CPI is $\text{CPI}_{pipe}$, then for the same clock rate: $\text{Speedup} \approx \frac{k}{\text{CPI}_{pipe}}$

In practice, clock rate differs because pipelining often changes cycle time; the most accurate analysis uses total execution time: $\text{Execution Time} = \text{Cycles} \times T_{clk}$

5) Pipeline utilization and pipeline efficiency (conceptual)

• Utilization is high when bubbles are rare.
• Efficiency decreases with increasing hazard-induced stalls/flushes.