Coursify

Network Theory

Block Coding & Hamming Distance

Block coding adds structured error-checking bits to a data word. The reliability of a block code relies heavily on its Minimum Hamming Distance ($d_{min}$), which defines the minimum number of bit flips required to accidentally turn one valid codeword into another valid codeword.

Learning Goals

  • Introduction to Block Coding: Mapping $k$-bit datawords into $n$-bit codewords ($n > k$).
  • Error Detection vs. Error Correction mechanisms in block codes.
  • Hamming Distance: Definition, calculating distance between two binary words, and Minimum Hamming Distance ($d_{min}$).
  • Geometric representation and conditions for detecting ($s$ errors) and correcting ($t$ errors).
  • Compute the Hamming distance between any two given binary sequences.
  • Mathematically prove how many errors a code can detect or correct using the formulas:$$d_{min} = s + 1$$$$d_{min} = 2t + 1$$

In the Data Link Layer of computer networks, block coding adds controlled redundancy so that transmission errors can be detected and, in some schemes, corrected before data is delivered upward.2 A source message is divided into kk-bit datawords and each dataword is mapped to an nn-bit codeword with n>kn>k. The added r=nkr=n-k bits carry no new payload information, but they create structure in the code space that helps the receiver recognize corruption.2

A central metric in this structure is the Hamming distance, defined as the number of differing bit positions between two binary words of equal length.2 For binary vectors xx and yy, it can be computed by XOR and bit counting:

d(x,y)=w(xy),d(x,y)=w(x\oplus y),

where w()w(\cdot) is the number of 11s in the result, also called the Hamming weight.2 The design quantity that matters most is the minimum Hamming distance dmind_{min}, because it determines how many errors the code can reliably detect or correct.3

In network-theory context, block coding is not merely abstract algebra: it expresses the tradeoff among reliability, bandwidth efficiency, and decoder complexity. A higher code rate k/nk/n preserves more bandwidth, while a larger dmind_{min} improves resilience to noise. This section develops the mapping from datawords to codewords, the difference between error detection and error correction, the geometry of Hamming space, and the formal results

dmin=s+1anddmin=2t+1,d_{min}=s+1 \qquad\text{and}\qquad d_{min}=2t+1,

which characterize guaranteed detection of ss errors and guaranteed correction of tt errors.3

Footnotes

  1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions. 2 3 4 5

  2. Data Link Layer - Data link layer notes explaining datawords, codewords, redundancy, and minimum-distance rules. 2 3 4 5

  3. Introduction to binary block codes - MIT material on Hamming space, spheres, and geometric interpretation of block codes.

  4. Minimum Hamming Distance - GeeksforGeeks - Accessible explanation of Hamming weight, Hamming distance, and examples. 2

  5. Codewords and Hamming Distance • Error Detection: parity - MIT - MIT notes summarizing how minimum distance determines detectable and correctable errors. 2 3

Hamming Distance and Minimum Hamming Distance

Why this matters in the Data Link Layer

Frames can be corrupted by noise, interference, attenuation, or synchronization issues. Block coding adds structured redundancy so a receiver can detect invalid bit patterns and, in stronger codes, infer the most likely transmitted codeword.2

Footnotes

  1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions.

  2. Data Link Layer - Data link layer notes explaining datawords, codewords, redundancy, and minimum-distance rules.

Block Coding Model

A block code can be described as an (n,k)(n,k) scheme: each kk-bit dataword is transformed into an nn-bit codeword, where n=k+rn=k+r and rr is the number of redundant bits. Because there are 2k2^k possible datawords, an encoder must assign one valid codeword to each of those 2k2^k inputs. Not every nn-bit word is valid; only the selected set of codewords belongs to the code. This restriction is what makes error detection possible.2

For example, consider a simple (3,2)(3,2) code:

DatawordCodeword
0000000000
0101011011
1010101101
1111110110

Only four of the eight possible 33-bit patterns are valid. If the receiver gets 001001, it can immediately declare an error because 001001 is not one of the legal codewords. This is the essence of error detection.

A stronger code may also support error correction by choosing the valid codeword nearest to the received word in Hamming distance.2 In that case, codewords must be separated more widely so that the “regions of influence” around them do not overlap.2

A useful rate measure is

Code rate=kn,\text{Code rate}=\frac{k}{n},

which quantifies efficiency. Higher redundancy lowers rate but can increase dmind_{min} and therefore reliability.

Footnotes

  1. Data Link Layer - Data link layer notes explaining datawords, codewords, redundancy, and minimum-distance rules. 2 3 4

  2. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions. 2 3

  3. Codewords and Hamming Distance • Error Detection: parity - MIT - MIT notes summarizing how minimum distance determines detectable and correctable errors. 2

  4. Introduction to binary block codes - MIT material on Hamming space, spheres, and geometric interpretation of block codes.

How to Compute Hamming Distance Between Two Binary Sequences

  1. 1
    Step 1

    The two binary sequences must have the same number of bits; otherwise the Hamming distance is not defined in the standard block-coding sense.2

    Footnotes

    1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions.

    2. Minimum Hamming Distance - GeeksforGeeks - Accessible explanation of Hamming weight, Hamming distance, and examples.

  2. 2
    Step 2

    Inspect each bit pair from left to right and note every position at which the bits differ.

    Footnotes

    1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions.

  3. 3
    Step 3

    XOR the two sequences. Every resulting 11 marks a differing position, while every 00 marks a matching position.2

    Footnotes

    1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions.

    2. Data Link Layer - Data link layer notes explaining datawords, codewords, redundancy, and minimum-distance rules.

  4. 4
    Step 4

    The total number of differing positions is the Hamming distance.2

    Footnotes

    1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions.

    2. Minimum Hamming Distance - GeeksforGeeks - Accessible explanation of Hamming weight, Hamming distance, and examples.

  5. 5
    Step 5

    A larger distance means the two words are more separated in Hamming space, which generally improves distinguishability under noise.2

    Footnotes

    1. Introduction to binary block codes - MIT material on Hamming space, spheres, and geometric interpretation of block codes.

    2. Codewords and Hamming Distance • Error Detection: parity - MIT - MIT notes summarizing how minimum distance determines detectable and correctable errors.

Worked Examples of Hamming Distance

Let us compute distances directly.

Example 1
For x=000x=000 and y=011y=011:

xy=000011=011x\oplus y = 000\oplus 011 = 011

The XOR result has two 11s, so

d(000,011)=2.d(000,011)=2.

This example is standard in block-coding discussions.2

Example 2
For x=10101x=10101 and y=11110y=11110:

1010111110=0101110101\oplus 11110 = 01011

The XOR result contains three 11s, hence

d(10101,11110)=3.d(10101,11110)=3.

Again, the distance equals the number of differing bit positions.2

Example 3
For x=1100101x=1100101 and y=1001100y=1001100:

11001011001100=01010011100101\oplus 1001100 = 0101001

The XOR result has three 11s, so

d(1100101,1001100)=3.d(1100101,1001100)=3.

For an entire code, we calculate all pairwise distances among valid codewords and choose the smallest. That smallest value is dmind_{min}.2 If a code has codewords {000,011,101,110}\{000,011,101,110\}, the pairwise distances are all 22, so

dmin=2.d_{min}=2.

In this code, every valid codeword is separated from every other valid codeword by at least two bit changes, so any single-bit corruption moves a transmitted codeword to an invalid word rather than another valid one.

Footnotes

  1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions. 2 3

  2. Data Link Layer - Data link layer notes explaining datawords, codewords, redundancy, and minimum-distance rules. 2 3 4

Fast exam shortcut

To compute Hamming distance quickly, XOR the two bit strings and count the 11s. This avoids manual comparison and directly matches the formal definition d(x,y)=w(xy)d(x,y)=w(x\oplus y).2

Footnotes

  1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions.

  2. Minimum Hamming Distance - GeeksforGeeks - Accessible explanation of Hamming weight, Hamming distance, and examples.

Minimum Hamming Distance and Its Meaning

The minimum distance dmind_{min} is the most important design parameter of a block code because the worst-separated pair of valid codewords determines the code’s guaranteed performance.2 Formally, if CC is the set of valid codewords, then

dmin=minx,yCxyd(x,y).d_{min}=\min_{\substack{x,y\in C\\x\ne y}} d(x,y).

Interpretation:

  • If valid codewords are too close, a small number of bit flips may transform one valid codeword into another, making reliable detection impossible.
  • If valid codewords are far apart, more corruption is needed before ambiguity arises.2

A classic relation states:

  • To detect up to ss errors in all cases, a code must satisfy dmins+1.d_{min}\ge s+1.
  • To correct up to tt errors in all cases, a code must satisfy dmin2t+1.d_{min}\ge 2t+1.

These are often written in exact guaranteed-capability form as

s=dmin1andt=dmin12.s=d_{min}-1 \qquad\text{and}\qquad t=\left\lfloor\frac{d_{min}-1}{2}\right\rfloor.

The equalities dmin=s+1d_{min}=s+1 and dmin=2t+1d_{min}=2t+1 describe the threshold values for exact guaranteed detection and correction limits.2

This explains familiar cases:

dmind_{min}Guaranteed detectionGuaranteed correction
1100 errors00 errors
2211 error00 errors
3322 errors11 error
4433 errors11 error
5544 errors22 errors

The correction count grows more slowly because correction requires not only noticing that an error occurred, but deciding which original codeword was sent.2

Footnotes

  1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions. 2 3 4

  2. Data Link Layer - Data link layer notes explaining datawords, codewords, redundancy, and minimum-distance rules.

  3. Codewords and Hamming Distance • Error Detection: parity - MIT - MIT notes summarizing how minimum distance determines detectable and correctable errors. 2 3

  4. Introduction to binary block codes - MIT material on Hamming space, spheres, and geometric interpretation of block codes.

Guaranteed Capability vs. Minimum Hamming Distance

Comparison of detectable and correctable error counts implied by dmind_{min}.2

Footnotes

  1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions.

  2. Codewords and Hamming Distance • Error Detection: parity - MIT - MIT notes summarizing how minimum distance determines detectable and correctable errors.

Geometric View: Hamming Space, Spheres, and Decoding Regions

All binary words of length nn can be viewed as points in an nn-dimensional Hamming space.2 Valid codewords occupy selected points in this space, and the Hamming distance measures how many coordinate changes are needed to move from one point to another.

For correction, we imagine a Hamming sphere of radius tt around each valid codeword. Every received word inside that sphere is decoded to the center codeword.2 To guarantee correction of tt errors, spheres of radius tt around distinct codewords must not overlap. If they overlapped, some received word would be within distance tt of two different codewords, causing ambiguity.2

For detection only, overlap is not the issue. Instead, we need every pattern of up to ss bit errors to move the transmitted codeword outside the set of all valid codewords.2 Therefore, no two codewords may be closer than s+1s+1.

This geometric picture is especially valuable because it turns an algebraic rule into an intuitive one:

  • Detection means corrupted words should not land on another legal codeword.
  • Correction means corrupted words should remain closest to the true codeword.2

Footnotes

  1. Introduction to binary block codes - MIT material on Hamming space, spheres, and geometric interpretation of block codes. 2 3 4 5

  2. Hamming Metric and the Minimum Distance - UCSD notes providing proof ideas for correction capability via triangle inequality and Hamming spheres. 2 3

  3. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions. 2

  4. Data Link Layer - Data link layer notes explaining datawords, codewords, redundancy, and minimum-distance rules.

  5. Codewords and Hamming Distance • Error Detection: parity - MIT - MIT notes summarizing how minimum distance determines detectable and correctable errors.

Proof Idea for Error Detection: Why $d_{min}=s+1$

  1. 1
    Step 1

    This means every pair of distinct valid codewords differs in at least dmind_{min} bit positions.2

    Footnotes

    1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions.

    2. Data Link Layer - Data link layer notes explaining datawords, codewords, redundancy, and minimum-distance rules.

  2. 2
    Step 2

    Suppose a codeword xx is sent and at most ss bit errors occur during transmission.

  3. 3
    Step 3

    The received word yy differs from xx in at most ss positions, so d(x,y)sd(x,y)\le s.

  4. 4
    Step 4

    If yy were some other valid codeword zexz e x, then d(x,z)d(x,z) would have to be at most ss, contradicting the definition that all distinct codewords are at least dmind_{min} apart.

  5. 5
    Step 5

    Thus guaranteed detection of all patterns of up to ss errors requires s<dmins<d_{min}, equivalently dmins+1d_{min}\ge s+1.2

    Footnotes

    1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions.

    2. Codewords and Hamming Distance • Error Detection: parity - MIT - MIT notes summarizing how minimum distance determines detectable and correctable errors.

Proof Idea for Error Correction: Why $d_{min}=2t+1$

  1. 1
    Step 1

    The decoder selects the valid codeword with smallest Hamming distance from the received word, which is the standard minimum-distance decoding rule.2

    Footnotes

    1. Codewords and Hamming Distance • Error Detection: parity - MIT - MIT notes summarizing how minimum distance determines detectable and correctable errors.

    2. Hamming Metric and the Minimum Distance - UCSD notes providing proof ideas for correction capability via triangle inequality and Hamming spheres.

  2. 2
    Step 2

    If codeword xx is sent and the received word is yy, then d(x,y)td(x,y)\le t.

  3. 3
    Step 3

    For successful correction, no other valid codeword zz can also lie within distance tt of yy.

  4. 4
    Step 4

    If both d(x,y)td(x,y)\le t and d(z,y)td(z,y)\le t, then d(x,z)d(x,y)+d(y,z)2td(x,z)\le d(x,y)+d(y,z)\le 2t.

  5. 5
    Step 5

    But distinct codewords must be separated by at least dmind_{min}. Therefore ambiguity is impossible only when dmin>2td_{min}>2t, equivalently dmin2t+1d_{min}\ge 2t+1.2

    Footnotes

    1. Codewords and Hamming Distance • Error Detection: parity - MIT - MIT notes summarizing how minimum distance determines detectable and correctable errors.

    2. Hamming Metric and the Minimum Distance - UCSD notes providing proof ideas for correction capability via triangle inequality and Hamming spheres.

  6. 6
    Step 6

    Correction is harder than detection because the decoder must identify the original word, not just flag inconsistency.

Common Questions and Edge Cases

For binary words xx and yy of equal length,

d(x,y)=w(xy)d(x,y)=w(x\oplus y)

where w()w(\cdot) counts the number of 11s in the XOR result.2

Footnotes

  1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions.

  2. Minimum Hamming Distance - GeeksforGeeks - Accessible explanation of Hamming weight, Hamming distance, and examples.

Conceptual Roadmap for Learning Block Coding and Hamming Distance

Represent data in fixed-size blocks

Stage 1

Divide the bit stream into kk-bit datawords so encoding can be applied systematically."

Footnotes

  1. Data Link Layer - Data link layer notes explaining datawords, codewords, redundancy, and minimum-distance rules.

Add redundancy

Stage 2

Map each dataword to an nn-bit codeword with n>kn>k, introducing structure into the set of valid words."

Footnotes

  1. Data Link Layer - Data link layer notes explaining datawords, codewords, redundancy, and minimum-distance rules.

Measure pairwise separation

Stage 3

Use Hamming distance to quantify how far apart codewords are in binary space.2"

Footnotes

  1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions.

  2. Minimum Hamming Distance - GeeksforGeeks - Accessible explanation of Hamming weight, Hamming distance, and examples.

Find $d_{min}$

Stage 4

Compute the smallest pairwise distance among all valid codewords; this determines guaranteed capability.2"

Footnotes

  1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions.

  2. Data Link Layer - Data link layer notes explaining datawords, codewords, redundancy, and minimum-distance rules.

Apply detection and correction conditions

Stage 5

Use dmins+1d_{min}\ge s+1 for detection and dmin2t+1d_{min}\ge 2t+1 for correction.3"

Footnotes

  1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions.

  2. Codewords and Hamming Distance • Error Detection: parity - MIT - MIT notes summarizing how minimum distance determines detectable and correctable errors.

  3. Hamming Metric and the Minimum Distance - UCSD notes providing proof ideas for correction capability via triangle inequality and Hamming spheres.

Interpret geometrically

Stage 6

View valid codewords as centers of Hamming spheres whose overlap properties determine correctability.2"

Footnotes

  1. Introduction to binary block codes - MIT material on Hamming space, spheres, and geometric interpretation of block codes.

  2. Hamming Metric and the Minimum Distance - UCSD notes providing proof ideas for correction capability via triangle inequality and Hamming spheres.

Frequent misconception

A code that can detect 22 errors does not automatically correct 11 error and detect 22 errors simultaneously under all decoding strategies. For example, correcting 11 error requires dmin3d_{min}\ge 3, but correcting 11 and also safely distinguishing some larger patterns may need stronger constraints depending on the decoder and objective.

Footnotes

  1. Codewords and Hamming Distance • Error Detection: parity - MIT - MIT notes summarizing how minimum distance determines detectable and correctable errors.

Network-Theory Interpretation

Within the data link layer, block coding supports reliable frame delivery across imperfect physical media.2 Although many practical systems use specialized error-detecting codes such as CRC for frame checking, the theory of block coding and Hamming distance provides the mathematical foundation for understanding why redundancy works at all.2 The key ideas transfer broadly:

  • legal patterns are separated in code space,
  • channel noise perturbs transmitted patterns,
  • receiver logic exploits structure to detect or correct perturbations.2

From a design perspective, block coding creates a tradeoff:

More redundancylower rate knpotentially larger dminbetter reliability.\text{More redundancy} \Rightarrow \text{lower rate } \frac{k}{n} \Rightarrow \text{potentially larger } d_{min} \Rightarrow \text{better reliability}.

This is why coding theory sits naturally inside network performance analysis: it connects bandwidth, noise tolerance, and decoding certainty.

A concise summary is:

ConceptMeaningKey formula
Block codingMap kk information bits to nn coded bitsn=k+rn=k+r
Hamming distanceNumber of differing positions between two wordsd(x,y)=w(xy)d(x,y)=w(x\oplus y)
Minimum distanceWorst-case separation among valid codewordsdmin=mind(x,y)d_{min}=\min d(x,y)
Error detectionKnow an error occurreddmins+1d_{min}\ge s+1
Error correctionRecover original codeworddmin2t+1d_{min}\ge 2t+1

For this module, you should be able to compute distances between binary sequences, determine dmind_{min} from a codebook, and prove or apply the thresholds for detection and correction.3

Footnotes

  1. Data Link Layer - Akshay Jain - Lecture notes covering block coding, Hamming distance, minimum distance, and detection/correction conditions. 2 3

  2. Data Link Layer - Data link layer notes explaining datawords, codewords, redundancy, and minimum-distance rules. 2 3

  3. Introduction to binary block codes - MIT material on Hamming space, spheres, and geometric interpretation of block codes.

  4. Codewords and Hamming Distance • Error Detection: parity - MIT - MIT notes summarizing how minimum distance determines detectable and correctable errors. 2 3

Knowledge Check

Question 1 of 5
Q1Single choice

In a block code, what does the notation (n,k)(n,k) mean?

Block Coding & Hamming Distance | Network Theory | Coursify