OUTCORA Learning

Forward testing vs backtesting probability forecasts

A historical replay and a forward record can answer different questions. For a probability forecast, the most important distinction is when the estimate was fixed relative to the event outcome.

Historical backtesting

A backtest applies a defined method to past data. It helps examine model behaviour across historical periods, but its validity depends on the inputs that would actually have been available at each past observation time. Using information published later, selecting parameters after seeing the results, or silently excluding difficult intervals can make the replay appear stronger than it was.

Forward research

In forward research, observations are recorded as new public reference and event-market data arrive. A meaningful record fixes the model estimate, observed market figure, timestamp, market identifier and model version before the outcome is known. It then appends the official settlement outcome when available. This sequence makes it possible to distinguish a contemporaneous forecast from a reconstruction made after the fact.

What to keep in an evaluation ledger

Retain the full eligible observation set and the selection rule, including cases excluded by data-quality checks and the reason for each exclusion. Distinguish observations, signals and theoretical paper positions; they are different counts. When publishing a result, show the start and end dates, sample size, missing data treatment, model version, outcome source and any assumptions about hypothetical fills, spread and fees.

Forward does not mean executed

A forward record can use live incoming data while still remaining theoretical. A paper position is not a user transaction, and its apparent gain or loss is not a realised customer return. The method can be tested against later outcomes, but a positive historical or forward-research result does not guarantee future accuracy. OUTCORA provides analysis only and does not place transactions or handle client funds.