ECE gap
Compare expected calibration error across groups.
- Testing group
- Fairness
- Applies to
- Calibration within groups
- App version
2.0.0- Channel
- stable
Required inputs
Predictions and reference labels with the group attributes needed for the selected comparison.
Test procedure
Define the groups and task settings, compute the selected comparison, and inspect each group's underlying results.
Interpreting results
Read gaps alongside group sizes, errors and the task's harms. Different fairness measures answer different questions.
Limitations
Evidence record
Record the group definition, selected measure, thresholds and group results with the interpretation and remaining questions.