Targets for Code Optimization: Time, Space, and System-Level Examples

Code optimization is not just about making programs “faster”; it is about improving a chosen set of objectives (targets) under constraints (correctness, compatibility, energy, maintainability). A useful mental model is that compilers and runtimes trade between time (how many instructions/operations execute and how long they take) and space (code size, registers, temporaries, and allocations). For embedded contexts, minimizing code size can be a first-class target; for general-purpose systems, minimizing latency and maximizing throughput often dominate. In practice, optimization targets show up as compiler passes (e.g., dead code elimination, common subexpression elimination, loop transformations) and as machine-level strategies (e.g., locality, instruction working set size). 3

Below are the main targets you can optimize toward, with concrete examples and what “success” looks like.

Footnotes

1. Compiler Optimization - an overview - Survey-style overview touching core compiler optimization categories and related tooling. ↩

2. Code size optimization for embedded processors (UCAM-CL-TR-607) - Discusses dead/unreachable code elimination and frames time/space resources for code optimization. ↩

3. Intel CPU Metrics Reference - Defines metrics like front-end latency and L2 bound to diagnose stall sources. ↩

Optimization target taxonomy

A. Time-related targets

• Instruction count reduction (fewer operations)
• Lower latency (reduce stalls, misses)
• Higher throughput / higher IPC (more useful work per cycle)

B. Space-related targets

• Code size (binary size, instruction cache pressure)
• Register/temporary pressure (less spilling; fewer loads/stores)
• Heap allocation reduction (fewer allocations; less GC pressure)

C. System-level targets

• Energy / thermal constraints (often correlated with stalls and work done)
• Cache/memory behavior (spatial locality, reduced cache misses)
• Front-end behavior (instruction-cache misses / decode bandwidth constraints)

These are not mutually exclusive: e.g., loop unrolling targets time by reducing loop overhead but explicitly increases binary size (a space–time tradeoff).

Footnotes

1. Loop unrolling - Wikipedia - Explains loop unrolling as a space–time tradeoff and notes modern cache effects. ↩

Key time-related targets with examples

1) Reduce instruction count (less work executed)

A compiler can eliminate computations or blocks that do not affect observable results—this reduces the number of executed instructions.

Dead code elimination (DCE) removes code that is unreachable or has no effect on the output. Unreachable-code removal is explicitly discussed as part of dead/unreachable code elimination in compiler literature.

Example (unreachable debug branch)

1int f(int x) {
2  return x * 2;
3  // unreachable:
4  if (0) { x = x + 1; }
5}

After DCE, the unreachable branch disappears; the function becomes simply:

• compute $x*2$ and return.

2) Reduce redundant computations (common subexpression elimination)

If an expression like a+b is computed multiple times without changing operands, a compiler can compute it once and reuse the value. This targets time by removing repeated instructions (and sometimes enables further constant propagation). (This idea is commonly listed among core compiler optimizations alongside DCE and folding.)

Example

1int g(int a, int b, int c) {
2  int t1 = a + b;
3  int t2 = a + b;   // redundant
4  return t1 + c + t2;
5}

Optimized form:

• compute a+b once, reuse it for t2.

3) Improve latency by reducing stalls (front-end and memory stalls)

Modern CPUs spend many cycles waiting: for instructions to arrive (instruction-cache / front-end latency) or for data to arrive (cache misses). Intel’s performance guidance describes metrics such as front-end latency and L2 bound—both are stall sources you can reduce by shrinking working sets and improving code/layout and locality.

What helps

• Reduce instruction working set size (code size / layout)
• Improve data locality (sequential access, reuse)
• Reduce branches that cause misprediction (often affects front-end)

4) Increase throughput / utilization (raise IPC by feeding the core)

Higher IPC generally means the CPU retires more useful instructions per cycle. A practical demonstration of how locality changes measured performance can be seen in experiments comparing sequential vs random access, where cache-miss rate dramatically changes and runtime/cycle counts follow.

Example: sequential vs random access

• Sequentially walking an array tends to use spatial locality, causing far fewer cache misses.
• Random access increases cache misses and increases time due to waiting on memory.

This is a time target achieved indirectly through cache behavior.

Footnotes

1. Code size optimization for embedded processors (UCAM-CL-TR-607) - Discusses dead/unreachable code elimination and frames time/space resources for code optimization. ↩

2. Compiler Optimization - an overview - Survey-style overview touching core compiler optimization categories and related tooling. ↩

3. Intel CPU Metrics Reference - Defines metrics like front-end latency and L2 bound to diagnose stall sources. ↩

4. From Concepts to Reality: Measuring Throughput, Cache Misses, CPU Behavior in C++ - Shows sequential vs random access experiments with cache-miss-rate effects on runtime/cycles. ↩

Space-related targets with examples

5) Reduce code size (binary size) for instruction cache / embedded constraints

Code size is a distinct target, especially for embedded processors where memory/flash is limited. Compiler research explicitly frames resources for code as both time (schedule of instructions) and space (registers and temporaries), and discusses code size optimization in that context. 2

Example: instruction-size selection Intel’s Optimization Reference Manual includes guidance like using 32-bit instruction forms in 64-bit mode “to reduce code size unless the 64-bit version is necessary,” and notes instruction working set impacts (front-end behavior).

In compiler terms, this can translate to:

• use smaller encodings when semantically safe
• avoid unnecessary widening/narrowing that forces larger instructions

6) Reduce register pressure and spilling (fewer loads/stores)

When expressions require too many live temporaries, compilers may spill values to memory (increasing instruction count and latency). While the specific pass names vary by compiler backend, the target is consistent: reduce the number of simultaneously live values and enable better scheduling, so values stay in registers.

A practical symptom:

• More stack traffic (loads/stores)
• Higher memory-related stalls

This target often interacts with loop transformations (because unrolling can increase register pressure).

7) Reduce allocations (heap use) and GC pressure in managed/JIT runtimes

JIT compilers often target allocation reduction using escape analysis, which can decide that an object does not escape a scope and thus allow “scalar replacement” (representing the object as separate values) instead of allocating on the heap. This reduces allocations and can eliminate GC work. 2

Example pattern (allocation eliminated)

1for (int i = 0; i < 10_000_000; i++) {
2    Rectangle rect = new Rectangle(i + 5, i + 10); // may be optimized away
3    sum += rect.getArea();
4}

With successful escape analysis and scalar replacement, the allocation can be eliminated because the object’s lifetime and usage stay local to the loop.

Tradeoff

• Aggressive optimization may increase compile time or require profile-driven decisions.

Footnotes

1. Compiler Optimization - an overview - Survey-style overview touching core compiler optimization categories and related tooling. ↩

2. Code size optimization for embedded processors (UCAM-CL-TR-607) - Discusses dead/unreachable code elimination and frames time/space resources for code optimization. ↩

3. Intel(R) 64 and IA-32 Architectures Optimization Reference Manual (PDF) - Includes explicit rules for reducing code size and instruction working set concerns. ↩

4. JIT Compiler, Inlining and Escape Analysis - JVM Advent - Explains inlining and escape analysis with examples such as allocation elimination/scalar replacement. ↩ ↩²

5. Escape! From the Allocations: Escape Analysis (kipp.ly) - Describes how escape analysis can eliminate allocations and discuss related optimization ideas. ↩

System-level targets: locality and instruction working set

8) Optimize cache locality (reduce cache misses)

Cache misses often dominate runtime when working sets don’t fit in cache. Cache locality is strongly connected to how you order memory accesses. Experiments comparing sequential vs random access show large differences in cache-miss rates and corresponding runtime/cycles.

Example: loop order If you have a 2D array A[row][col], accessing in column-major order (when stored row-major) can cause many more cache misses than row-wise traversal.

9) Optimize instruction-cache / front-end latency

Even if you do the same “amount” of computation, larger binaries can increase instruction-cache misses or decode pressure. Intel’s manual discusses how keeping instruction working set footprint small can help the front end take advantage of optimal decode bandwidth on certain cores.

This directly connects with the known space–time tradeoff of loop unrolling:

• unrolling can speed up (fewer loop-control instructions, more ILP)
• but it inflates code size and may increase cache misses, making it counterproductive on modern CPUs.

Footnotes

1. From Concepts to Reality: Measuring Throughput, Cache Misses, CPU Behavior in C++ - Shows sequential vs random access experiments with cache-miss-rate effects on runtime/cycles. ↩

2. Intel(R) 64 and IA-32 Architectures Optimization Reference Manual (PDF) - Includes explicit rules for reducing code size and instruction working set concerns. ↩

3. Loop unrolling - Wikipedia - Explains loop unrolling as a space–time tradeoff and notes modern cache effects. ↩

Mini-examples: mapping targets → typical compiler/runtime actions

Quick checklist: “Which target am I optimizing?”

• If you see lots of unnecessary instructions: aim at instruction count (DCE, CSE).
• If runtime is dominated by stalls: aim at latency via cache/locality and front-end behavior.
• If you’re in embedded / size-constrained environments: aim at code size and sometimes instruction encoding choices.
• If you see allocation/GC overhead: aim at allocation elimination via escape analysis (JIT) / reducing temporaries (native).
• If loop hotspots dominate: consider loop transformations, but verify the space–time tradeoff (unrolling can hurt due to code size/cache).

Footnotes

1. Intel(R) 64 and IA-32 Architectures Optimization Reference Manual (PDF) - Includes explicit rules for reducing code size and instruction working set concerns. ↩

2. JIT Compiler, Inlining and Escape Analysis - JVM Advent - Explains inlining and escape analysis with examples such as allocation elimination/scalar replacement. ↩

3. Loop unrolling - Wikipedia - Explains loop unrolling as a space–time tradeoff and notes modern cache effects. ↩