Fairness and limitations
Introduced
Vocabulary
| English | Meaning |
|---|---|
| error disparity | A difference in model error rates between groups. |
Per-group error
- Error disparity: compare per-group confusion matrices. Overnight's two regions: city recall 27/30 = 0.90; rural 9/12 = 0.75 — a 15-point gap.
{exercise:uni-cs1307-disparity}
Practice
City recall 0.90, rural 0.75: the disparity is —
Difficulty 1/5 · 100 coins + a small bonus on first completion today
0.90 - 0.75: one in four rural lates missed vs one in ten.
The parity arithmetic}
- Rural lates are 13% of the region's records against the city's 29%: the minority class is thinner, and the boundary under-serves it. The fix starts in the data (more records, class weights), not in the metric.
- Fairness criteria have names and conflicts: equal-error-rate parity separates thresholds; demographic parity equalises positive rates — sometimes mutually impossible. Fairness is a design choice, not a checkbox.
{exercise:uni-cs1307-parity}
Practice
The rural gap most likely starts in —
Difficulty 1/5 · 100 coins + a small bonus on first completion today
13% vs 29% class thinness: fix the data, not the number.
The limitation statement
- Overnight is synthetic, 200 records, two regions, one season: its 0.92 claims nothing about your city or carrier. The model card states intended use, measured disparities (the 15-point gap), and out-of-scope uses.
- Declaring fairness by overall accuracy hides the gap underneath; a limitation honestly stated is trust — one discovered by users is the opposite.
{exercise:uni-cs1307-limits}
Practice
The model card limitation section exists to —
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Honestly stated limits are trust; user-discovered ones are the opposite.