Fairness and limitations
Introduit
Vocabulaire
| English | Sens |
|---|---|
| error disparity | A difference in model error rates between groups. |
English
Per-group error
- Error disparity: compare per-group confusion matrices. Overnight's two regions: city recall 27/30 = 0.90; rural 9/12 = 0.75 — a 15-point gap.
{exercise:uni-cs1307-disparity}
Pratique
City recall 0.90, rural 0.75: the disparity is —
Difficulté 1/5 · 100 pièces + un petit bonus lors de la première réussite aujourd'hui
0.90 - 0.75: one in four rural lates missed vs one in ten.
English
The parity arithmetic}
- Rural lates are 13% of the region's records against the city's 29%: the minority class is thinner, and the boundary under-serves it. The fix starts in the data (more records, class weights), not in the metric.
- Fairness criteria have names and conflicts: equal-error-rate parity separates thresholds; demographic parity equalises positive rates — sometimes mutually impossible. Fairness is a design choice, not a checkbox.
{exercise:uni-cs1307-parity}
Pratique
The rural gap most likely starts in —
Difficulté 1/5 · 100 pièces + un petit bonus lors de la première réussite aujourd'hui
13% vs 29% class thinness: fix the data, not the number.
English
The limitation statement
- Overnight is synthetic, 200 records, two regions, one season: its 0.92 claims nothing about your city or carrier. The model card states intended use, measured disparities (the 15-point gap), and out-of-scope uses.
- Declaring fairness by overall accuracy hides the gap underneath; a limitation honestly stated is trust — one discovered by users is the opposite.
{exercise:uni-cs1307-limits}
Pratique
The model card limitation section exists to —
Difficulté 1/5 · 100 pièces + un petit bonus lors de la première réussite aujourd'hui
Honestly stated limits are trust; user-discovered ones are the opposite.