t_000003 Predictive Accuracy Calibration (v17.0)

Summary

Task: Analyze model calibration and identify miscalibration patterns Status: ACTION_REQUIRED Calibration Acceptable: ⚠️ No

Metrics

• Brier Score: 0.2703
• Overall Accuracy: 50.0%
• Average Confidence: 55.0%
• Confidence-Accuracy Gap: +2.50%

Recommended Actions

1. temperature_scale

   • Action: decrease_confidence_scaling
   • Rationale: Model shows overconfidence gap of +2.50%. Increase temperature to soften confidence estimates.

2. area_specific

   • Action: retrain_on_more_samples
   • Rationale: Area area128 shows significant calibration gap: 50.0% confidence vs 0.0% accuracy

3. area_specific

   • Action: retrain_on_more_samples
   • Rationale: Area area129 shows significant calibration gap: 55.0% confidence vs 100.0% accuracy

4. area_specific

   • Action: retrain_on_more_samples
   • Rationale: Area area54 shows significant calibration gap: 50.0% confidence vs 100.0% accuracy

5. area_specific

   • Action: retrain_on_more_samples
   • Rationale: Area area73 shows significant calibration gap: 55.0% confidence vs 0.0% accuracy