Information and coding introduction
Đã giới thiệu| English | Ý nghĩa |
|---|---|
| entropy/ˈentrəpi/ | entropy |
| channel capacity | dung lượng kênh truyền |
| Hamming code | mã Hamming |
Surprise, measured
- Entropy $H = -\sum p\log_2 p$; Huffman meets it for dyadic sources. The quartile source: $H = 1.75$ bits, Huffman lengths 1,2,3,3 average exactly 1.75 ✓.
{exercise:uni-ee1107-entropy}
The quartile source (1/2, 1/4, 1/8, 1/8) has entropy —
Độ khó 1/5 · 100 xu + phần thưởng nhỏ khi hoàn thành lần đầu tiên hôm nay
The four p·log₂(1/p) terms are 0.5, 0.5, 0.375, 0.375: H = 1.75 bits; dyadic, so Huffman lengths 1,2,3,3 average exactly 1.75.
The ceiling
- Channel capacity $C = B\log_2(1+S/N)$: 3 kHz at 30 dB is 29.9 kbps — the dial-up ceiling, approached never beaten. The Hamming code (7,4) corrects any single error at 43% redundancy.
{exercise:uni-ee1107-ceiling}
A 3 kHz channel at 30 dB SNR has capacity about —
Độ khó 1/5 · 100 xu + phần thưởng nhỏ khi hoàn thành lần đầu tiên hôm nay
C = 3000·log₂(1001) = 3000×9.97 = 29.9 kbps — the dial-up modem ceiling, approached never beaten.
In Hamming (7,4), the syndrome after flipping bit 5 reads —
Độ khó 1/5 · 100 xu + phần thưởng nhỏ khi hoàn thành lần đầu tiên hôm nay
Each parity bit covers a syndrome bit: bit 5 touches parities 1 and 3 → syndrome 101 = 5. The code POINTS at the error.