What you will learn
- Rolling estimates
- Point-in-time inputs
- Error metrics
- Stress overrides as governance decisions
Rolling estimates
A rolling forecast must use only data available before its target period. Save the forecast when made, then compare it with the later realized measure. Recomputing old forecasts with revised inputs can create an unrealistically clean history.
Point-in-time inputs
Point-in-time inputs include publication and correction timing. A series stored today may contain values unavailable at the historical decision. Preserve data versions where revisions can affect evaluation.
Error metrics
Choose error metrics suited to the target and decision. Squared error emphasizes large errors; absolute error weights them differently. Do not select the metric only because it favors the preferred model.
Stress overrides as governance decisions
Stress overrides are governance choices that alter behavior when conditions fall outside ordinary assumptions. Define triggers, ownership and review rather than presenting an ad hoc override as part of an unchanged forecast model.
Worked example
Forecast A has one very large miss and many small misses; B has moderate misses throughout. Their ranking can depend on the error metric, so the relevant loss function should be specified before comparison.
Try it yourself
Write an evaluation protocol naming the target, forecast timestamp, error metric and handling of revised observations.
Show the worked solution
Use a fixed target definition and stored forecasts, select the metric for the decision objective, retain original data vintages and report revisions separately. This distinguishes prospective performance from retrospective reconstruction.
Apply this to your course project
Compare several volatility estimates and document where their forecasts fail.
Keep the calculation inputs, assumptions and decisions with your work. Practical exercises are self-reviewed; the scored knowledge checks assess the questions shown, not an independent certification of practical competence.