Brier Score

scoringrules.brier_score

brier_score(
    observations: ArrayLike,
    forecasts: ArrayLike,
    /,
    *,
    backend: Backend = None,
) -> Array

Compute the Brier Score (BS).

The BS is formulated as

\[ BS(f, y) = (f - y)^2, \]

where \(f \in [0, 1]\) is the predicted probability of an event and \(y \in \{0, 1\}\) the actual outcome.

Parameters:

Name	Type	Description	Default
`observations`	`ArrayLike`	Observed outcome, either 0 or 1.	required
`forecasts`	`NDArray`	Forecasted probabilities between 0 and 1.	required
`backend`	`Backend`	The name of the backend used for computations. Defaults to 'numpy'.	`None`

Returns:

Name	Type	Description
`brier_score`	`NDArray`	The computed Brier Score.

scoringrules.rps_score

rps_score(
    observations: ArrayLike,
    forecasts: ArrayLike,
    /,
    axis: int = -1,
    *,
    backend: Backend = None,
) -> Array

Compute the (Discrete) Ranked Probability Score (RPS).

Suppose the outcome corresponds to one of \(K\) ordered categories. The RPS is defined as

\[ RPS(f, y) = \sum_{k=1}^{K}(\tilde{f}_{k} - \tilde{y}_{k})^2, \]

where \(f \in [0, 1]^{K}\) is a vector of length \(K\) containing forecast probabilities that each of the \(K\) categories will occur, and \(y \in \{0, 1\}^{K}\) is a vector of length \(K\), with the \(k\)-th element equal to one if the \(k\)-th category occurs. We have \(\sum_{k=1}^{K} y_{k} = \sum_{k=1}^{K} f_{k} = 1\), and, for \(k = 1, \dots, K\), \(\tilde{y}_{k} = \sum_{i=1}^{k} y_{i}\) and \(\tilde{f}_{k} = \sum_{i=1}^{k} f_{i}\).

Parameters:

Name	Type	Description	Default
`observations`	`ArrayLike`	Array of 0's and 1's corresponding to unobserved and observed categories	required
`forecasts`	`ArrayLike`	Array of forecast probabilities for each category.	required
`axis`	`int`	The axis corresponding to the categories. Default is the last axis.	`-1`
`backend`	`Backend`	The name of the backend used for computations. Defaults to 'numpy'.	`None`

Returns:

Name	Type	Description
`score`	`Array`	The computed Ranked Probability Score.

scoringrules.log_score

log_score(
    observations: ArrayLike,
    forecasts: ArrayLike,
    /,
    *,
    backend: Backend = None,
) -> Array

Compute the Logarithmic Score (LS) for probability forecasts for binary outcomes.

The LS is formulated as

\[ LS(f, y) = -\log|f + y - 1|, \]

where \(f \in [0, 1]\) is the predicted probability of an event and \(y \in \{0, 1\}\) the actual outcome.

Parameters:

Name	Type	Description	Default
`observations`	`ArrayLike`	Observed outcome, either 0 or 1.	required
`forecasts`	`NDArray`	Forecasted probabilities between 0 and 1.	required
`backend`	`Backend`	The name of the backend used for computations. Defaults to 'numpy'.	`None`

Returns:

Name	Type	Description
`score`	`Array`	The computed Log Score.

scoringrules.rls_score

rls_score(
    observations: ArrayLike,
    forecasts: ArrayLike,
    /,
    axis: int = -1,
    *,
    backend: Backend = None,
) -> Array

Compute the (Discrete) Ranked Logarithmic Score (RLS).

Suppose the outcome corresponds to one of \(K\) ordered categories. The RLS is defined as

\[ RPS(f, y) = -\sum_{k=1}^{K} \log|\tilde{f}_{k} + \tilde{y}_{k} - 1|, \]

where \(f \in [0, 1]^{K}\) is a vector of length \(K\) containing forecast probabilities that each of the \(K\) categories will occur, and \(y \in \{0, 1\}^{K}\) is a vector of length \(K\), with the \(k\)-th element equal to one if the \(k\)-th category occurs. We have \(\sum_{k=1}^{K} y_{k} = \sum_{k=1}^{K} f_{k} = 1\), and, for \(k = 1, \dots, K\), \(\tilde{y}_{k} = \sum_{i=1}^{k} y_{i}\) and \(\tilde{f}_{k} = \sum_{i=1}^{k} f_{i}\).

Parameters:

Name	Type	Description	Default
`observations`	`ArrayLike`	Observed outcome, either 0 or 1.	required
`forecasts`	`NDArray`	Forecasted probabilities between 0 and 1.	required
`backend`	`Backend`	The name of the backend used for computations. Defaults to 'numpy'.	`None`

Returns:

Name	Type	Description
`score`	`Array`	The computed Ranked Logarithmic Score.