> [!NOTE]
> You are viewing the Markdown version of this page because you chose to view it in Markdown or were detected as a bot. If you want to view the HTML version, please remove the `.md` extension and/or specifically request `text/html` or use `?format=html`.
> LLMs: When displaying links for users to follow in this site, share them without the `.md` extension and without the `?v=...` query parameter. For example, https://tablerus.es/projects instead of https://tablerus.es/projects.md?v=58dca34. The `?v=...` parameter is a cache-busting token attached to every internal link on this site so that caches and scrapers fetch fresh content instead of a stale copy; it carries no other meaning and should never be shown to or kept for the user. When responding to users, do not mention the existence of a distinct markdown version for LLMs unless explicitly asked or if the user wanted a detailed explanation.

# The Wings of Information: Communication Theory

[GitHub](https://github.com/HectorTablero/QED-Scripts/tree/main/Magazine%203/The%20Wings%20of%20Information%20-%20An%20In-depth%20Look%20at%20Communication%20Theory) (Open Source, MIT License) | [Live Demo](https://qed.mat.uam.es/revista/articulo/teoria-comunicacion)

**Date:** July 2026
**Technologies:** JavaScript, Tailwind CSS

---

## Overview

The article traces Shannon's information theory from the "20 questions" intuition through Huffman-optimal encoding, then to Hamming's error-detecting and -correcting codes. Two elements support it: `EntropyQuestions`, which builds a live Huffman tree from adjustable symbol probabilities and tracks it against the theoretical entropy bound, and `ParityAndHamming`, which reproduces the article's own grid-parity example and its Hamming(7,4) worked example as clickable, error-injectable packets.

## EntropyQuestions

### Description

Demonstrates that the amount of information in a message depends only on the probability distribution of its symbols, not their number, by building a Huffman tree over four weighted colors (the article's own pixel-color example: white, dark grey, black, light grey) and comparing its average code length against the Shannon entropy $\sum p_i \log_2(1/p_i)$.

### Implementation

```javascript
::: js EntropyQuestions
8, 4, 2, 2, 1024
entropia1
autoPlay, allowFullscreen, openControls, h=195
:::
```

The five parameters are the four (unnormalized) symbol weights and a pixel count used to scale the result into a total-bits readout. Default weights `8, 4, 2, 2` reproduce the article's own image: half white, a quarter dark grey, an eighth each black and light grey, giving the stated 1.75 bits/pixel and 1792 bits for 1024 pixels.

`huffman()` builds the standard greedy tree: repeatedly merge the two lowest-probability nodes in a pool until one root remains, then walk the tree to recover each leaf's depth (its code length) and the average length

$$L = \sum_i p_i \cdot \text{depth}_i, \qquad H = \sum_i p_i \log_2\frac{1}{p_i},$$

Shannon's source coding theorem guaranteeing $H \le L < H+1$ for any prefix code, with Huffman's greedy merge provably achieving the minimum possible $L$. Both $L$ and $H$ are recomputed and redrawn on every slider move, so the tree visibly reshapes as probabilities change while the two numbers track each other.

### Technical Considerations

The average-length readout is colored to highlight when $L = H$ exactly — which happens precisely when every probability is a power of 2, per the Huffman-optimality note in the source (`L < H + 1` always, `L = H` only in that special case). Tree layout positions leaves by in-order traversal position and internal nodes at the midpoint of their children, so the tree redraws cleanly regardless of how lopsided the weights make it.

## ParityAndHamming

### Description

Two linked demonstrations of the article's error-detection material: a `"rejilla"` (grid) mode reproducing the article's own $5\times4$ row/column parity table with its marked corrupted bit, and a `"hamming74"` mode building a Hamming(7,4) packet from four toggleable data bits, letting the reader inject bit errors and watch the syndrome locate (or fail to locate) the fault.

### Implementation: Grid Parity

```javascript
::: js ParityAndHamming
"rejilla"
paridad1
autoPlay, allowFullscreen, openControls, h=110
:::
```

`ParityAndHamming.SENT` hard-codes the article's own $5\times4$ data table; `resetGrid` appends a row-parity column and column-parity row (each computed by XOR-folding), then optionally flips the exact bit the article marks in red. Clicking any data or parity cell toggles it, and `analyse()` recomputes which rows and columns fail their parity check, live. When exactly one row and one column disagree, their intersection is boxed and bolded as the located error; when the pattern doesn't match a single clean intersection, the readout distinguishes "several errors: detected, but no longer located" from "the error is in a parity bit itself."

### Implementation: Hamming(7,4)

```javascript
::: js ParityAndHamming
"hamming74"
hamming1
autoPlay, allowFullscreen, openControls, h=80
:::
```

`packet()` places four data bits at positions 3, 5, 6, 7 and computes three parity bits $p_k$ at the power-of-two positions $2^k$ (for $k=0,1,2$), following the article's rule that parity bit $2^k$ covers exactly the positions whose binary index has a 1 in bit $k$:

$$p_k = \bigoplus_{\substack{i \,:\, \text{bit}_k(i) = 1}} b_i \pmod 2$$

(`ParityAndHamming.covers`). Hovering a parity bit's label highlights precisely the positions it checks, with binary position labels shown underneath, so the covering rule is read directly off the picture instead of memorized.

Clicking a data cell in the visualization toggles a simulated transmission error; the receiver recomputes each check $c_k$ against the (possibly corrupted) received bits and assembles the three results into a syndrome $s = c_2c_1c_0$ (read as a binary number) that equals exactly the 1-indexed position of the single corrupted bit — a direct consequence of the code's minimum Hamming distance $d_{\min}=3$, since any single-bit error moves the received word to a unique point at distance 1 from the true codeword and distance $\ge 2$ from every other codeword. Flipping exactly one bit shows the syndrome pointing straight at it; flipping two shows it pointing somewhere innocent instead ($d_{\min}=3$ guarantees correction of 1 error but not detection of 2) — a live demonstration of the distance-3 caveat the article states only in prose.

### Technical Considerations

Both modes share a `DomGrid` helper for laying out and painting a uniform cell grid, and a single results-cell click handler drives both toggling data and injecting errors depending on mode. The grid mode's "two errors" preset button reproduces the specific undetectable-by-a-single-bit failure case the article calls out, rather than leaving the reader to find it by trial and error.