coverage_rate – Empirical coverage rate – share of OOS observations falling within the nominal-α interval.#

Back to density_metrics axis | Back to L5 | Browse all options

Operational op under axis density_metrics, sub-layer L5_A_metric_specification, layer l5. Standalone callable: mf.functions.coverage_rate.

Function signature#

mf.functions.coverage_rate(
    y_true: np.ndarray | pd.Series,
    y_lower: np.ndarray | pd.Series,
    y_upper: np.ndarray | pd.Series,
) -> float

Parameters#

name

type

default

constraint

description

y_true

`np.ndarray

pd.Series`

—

—

y_lower

`np.ndarray

pd.Series`

—

—

y_upper

`np.ndarray

pd.Series`

—

—

Returns#

float — scalar result.

Behavior#

Density-forecast metric coverage_rate. Should equal α (1 - α miscoverage) if the model is well-calibrated. Deviations indicate miscalibration: low coverage = intervals too narrow; high coverage = intervals too wide. Pair with interval_score to capture both calibration and sharpness.

When to use

Interval-calibration audits; reporting alongside interval_score.

In recipe context#

Set params.density_metrics = "coverage_rate" in the relevant layer to activate this op within a recipe:

# Layer L5 recipe fragment
params:
  density_metrics: coverage_rate

References#

  • macroforecast design Part 3, L5: ‘evaluation = (metric × benchmark × aggregation × decomposition × ranking).’

  • Gneiting & Raftery (2007) ‘Strictly Proper Scoring Rules, Prediction, and Estimation’, JASA 102(477): 359-378. (doi:10.1198/016214506000001437)

  • Gneiting & Katzfuss (2014) ‘Probabilistic Forecasting’, Annual Review of Statistics and Its Application 1: 125-151.