Machine Learning for Economic Evidence
EVIDENCE LAB · 12
Forecasting
& Evaluation.
A forecast is a disciplined statement about an uncertain future. Learn to construct competing models, protect a test sample and let out-of-sample evidence decide.
TRAINTESTSCORE
01 · FORECAST BENCHMARKS
Complexity must defeat
a simple benchmark.
Last observation
Tomorrow equals today. Essential benchmark for persistent series.
Ŷₜ₊₁ = Yₜ
Moving average
Average the most recent k observations to smooth short-run noise.
Ŷₜ₊₁ = ΣYₜ₋ⱼ/k
Exponential smoothing
Update the forecast using a weighted combination of new evidence and the previous forecast.
Fₜ₊₁ = αYₜ+(1−α)Fₜ
Linear projection
Extend an estimated deterministic trend, cautiously and within a defensible horizon.
Ŷₜ₊ₕ = â+b̂(t+h)
02 · ERROR METRICS
Score forecasts
where they were not trained.
Mean absolute error
MAE = mean|Y−Ŷ|
Easy to interpret in the original units and less dominated by large misses.
Root mean squared error
RMSE = √mean(Y−Ŷ)²
Penalises large errors more heavily and is sensitive to extreme forecast failures.
Mean absolute percentage error
MAPE = mean|e/Y|×100
Scale-free but unstable or undefined when actual values are near zero.
INTERACTIVE FORECAST TOURNAMENT
Train. Predict.
Let evidence choose.
Change the data-generating process, holdout length and candidate model. ECONORIA estimates forecasts and scores them only on the protected test period.
FORECAST VERDICT
03 · FORECAST DISCIPLINE
Do not hide uncertainty
behind a single number.
Prediction intervals
Intervals widen with horizon because shocks and parameter uncertainty accumulate.
Rolling evaluation
Repeat training and testing across several forecast origins rather than trusting one split.
Structural breaks
Regime changes can make yesterday’s best model tomorrow’s worst forecast.
