ens_brier

hydrostats.ens_metrics.ens_brier(fcst_ens=None, obs=None, threshold=None, ens_threshold=None, obs_threshold=None, fcst_ens_bin=None, obs_bin=None, adj=None)

Calculate the ensemble-adjusted Brier Score.

Range: 0 ≤ Brier ≤ 1, lower is better.

Parameters:
  • obs (1D ndarray) – Array of observations for each start date.
  • fcst_ens (2D ndarray) – Array of ensemble forecast of dimension n x M, where n = number of start dates and M = number of ensemble members.
  • threshold (float) – The threshold for an event (e.g. if the event is a 100 year flood, the streamflow value that a 100 year flood would have to exceed.
  • ens_threshold (float) – If different threshholds for the ensemble forecast and the observed data is desired, then this parameter can be set along with the ‘obs_threshold’ parameter to set different thresholds.
  • obs_threshold (float) – If different threshholds for the ensemble forecast and the observed data is desired, then this parameter can be set along with the ‘ens_threshold’ parameter to set different thresholds.
  • fcst_ens_bin (1D ndarray) – Binary array of observations for each start date. 1 for an event and 0 for a non-event.
  • obs_bin (2D ndarray) – Binary array of ensemble forecast of dimension n x M, where n = number of start dates and M = number of ensemble members. 1 for an event and 0 for a non-event.
  • adj (float or int) – A positive number representing ensemble size for which the scores should be adjusted. If None (default) scores will not be adjusted. This value can be ‘np.inf‘, in which case the adjusted (or fair) Brier scores will be calculated.
Returns:

Array of length with the ensemble-adjusted Brier scores. Length may not equal n if data has been removed due to NaN or Inf values.
Return type:

1D ndarray

Notes

NaN and inf treatment: If any value in obs or fcst_ens is NaN or inf, then the corresponding row in both fcst_ens (for all ensemble members) and in obs will be deleted.

Examples

>>> import numpy as np
>>> import hydrostats.ens_metrics as em
>>> np.random.seed(3849590438)  # For reproducibility

Creating an observed 1D array and an ensemble 2D array

>>> ensemble_array = (np.random.rand(15, 52) + 1) * 100  # 52 Ensembles
>>> observed_array = (np.random.rand(15) + 1) * 100

Computing the ensemble-adjusted Brier score between the ensemble mean and the observed data.

>>> print(em.ens_brier(obs=observed_array, fcst_ens=ensemble_array, threshold=175))
[0.08321006 0.05325444 0.53402367 0.45303254 0.02995562 0.08321006
 0.08321006 0.03698225 0.02366864 0.0625     0.04474852 0.71597633
 0.04474852 0.04474852 0.09467456]
>>> np.mean(em.ens_brier(obs=observed_array, fcst_ens=ensemble_array, threshold=175))
0.15919625246548325

When we manually create binary data we get the same result

>>> ensemble_array_bin = (ensemble_array > 175).astype(np.int)
>>> observed_array_bin = (observed_array > 175).astype(np.int)
>>> print(em.ens_brier(obs_bin=observed_array_bin, fcst_ens_bin=ensemble_array_bin))
[0.08321006 0.05325444 0.53402367 0.45303254 0.02995562 0.08321006
 0.08321006 0.03698225 0.02366864 0.0625     0.04474852 0.71597633
 0.04474852 0.04474852 0.09467456]
>>> np.mean(em.ens_brier(obs_bin=observed_array_bin, fcst_ens_bin=ensemble_array_bin))
0.15919625246548325

References

  • Ferro CAT, Richardson SR, Weigel AP (2008) On the effect of ensemble size on the discrete and continuous ranked probability scores. Meteorological Applications. doi: 10.1002/met.45
  • Stefan Siegert (2017). SpecsVerification: Forecast Verification Routines for Ensemble Forecasts of Weather and Climate. R package version 0.5-2. https://CRAN.R-project.org/package=SpecsVerification