5.21 Time-series forecasting

What this is and why it exists

Forecasting is supervised learning with time as the axis, and most of the machinery you have relied on for twenty topics quietly stops holding. Rows are not independent. The future must never be visible during training. And the default split — shuffle, take a fifth — produces a backtest that looks superb and a forecast that fails in its first week. This topic is the rules that change, the two families of method that both remain competitive, and the one validation scheme that keeps the numbers honest.

The vocabulary

• Stationarity — the property of a series whose behaviour is stable over time: no drifting level, no growing spread, no shifting seasonal shape.
• Trend — a persistent movement up or down.
• Seasonality — a pattern repeating on a fixed period: hour of day, day of week, month of year.
• Differencing — replacing each value by its change from an earlier one, to remove trend or seasonality.
• Autocorrelation — the correlation of a series with its own earlier values.
• Lag feature — a past value of the series, used as an input column.
• Horizon — how far ahead the forecast reaches.
• Rolling origin — a backtest in which the cut between past and future moves forward repeatedly.
• Naive baseline — repeating the last value, or the value one season ago.

The mental model

Begin with why the usual machinery misfires, because the scikit-learn user guide states the reason exactly. Time series data, it says, "is characterized by the correlation between observations that are near in time (autocorrelation)", while classical methods such as k-fold and shuffled splits "assume the samples are independent and identically distributed, and would result in unreasonable correlation between training and testing instances (yielding poor estimates of generalization error) on time series data". From which follows the instruction the whole topic rests on: evaluate the model "on the 'future' observations least like those that are used to train the model".

Decomposition comes first. Look at the series and separate what you can see: a level, a trend, one or more seasonal patterns, and what is left over. A series is stationary when its behaviour does not change over time, and almost no interesting series is — sales grow, temperatures cycle, traffic peaks on weekdays. Differencing is the standard repair: replace each value by its change from the previous point and a straight-line trend disappears; subtract the value one season earlier and the seasonal pattern goes with it. The statsmodels documentation puts the purpose plainly — the order of differences "is to achieve stationarity in the context of a stochastic trend or seasonality". Differencing is not free: it removes information along with the trend, and one difference too many adds noise. Difference until the series looks stable, and no further.

The classical family remains a strong baseline, and skipping it is the common mistake. Exponential smoothing forecasts a weighted average of the past in which the weights fade as you reach further back, with variants that add a trend component and a seasonal one; it is cheap, robust, and hard to beat on a short series. ARIMA and its seasonal form are the other half. The statsmodels model takes an order described as "The (p,d,q) order of the model for the autoregressive, differences, and moving average components" — read that as three counts: how many past values feed the prediction, how many times the series is differenced, and how many past errors are carried forward. The seasonal version adds "The (P,D,Q,s) order of the seasonal component of the model for the AR parameters, differences, MA parameters, and periodicity", which is the same three counts repeated at a season length, where "s is an integer strictly greater than one" — twelve for monthly data with a yearly cycle, seven for daily data with a weekly one.

On a series of two hundred points with strong seasonality and no external drivers, these models frequently win, and they win with a handful of settings rather than a hundred. Reach for machine learning when you have long history, many related series, or external information — weather, holidays, promotions — that a model of the series alone has no way to use.

The machine learning approach turns the series into a table, and constructing that table correctly is the whole trick. Each row is one time point. The columns are lags (the value one step back, one week back, one year back), rolling statistics over a trailing window, and calendar features — day of week, month, holiday flag. Then a gradient booster over that table, which handles the nonlinearity and the interactions without being told about them.

Two rules govern every column, and violating either produces the fantasy backtest this topic exists to prevent. Every feature must be computable from information available at the moment of the forecast. A rolling mean must be trailing, never centred, because a centred window contains points from after the forecast time. A category average must be computed from the training period only. And the horizon must be built into the lags. If you forecast seven days ahead, the most recent value you may use is seven days old — a model handed yesterday's value scores beautifully and cannot be run, because at prediction time yesterday has not happened yet.

Backtesting with a rolling origin is the validation scheme, and scikit-learn implements it directly. TimeSeriesSplit "is a variation of k-fold which returns first k folds as train set and the (k+1)th fold as test set", and the guide flags the difference that matters: "unlike standard cross-validation methods, successive training sets are supersets of those that come before them". It "adds all surplus data to the first training partition, which is always used to train the model", and the folds "must represent the same duration, in order to have comparable metrics across folds". Each fold trains on everything before a cut and tests on what follows; the cut then moves forward and it happens again, which is exactly how the model will be used in production.

One parameter deserves attention because it encodes the horizon rule. gap is documented as the "Number of samples to exclude from the end of each train set before the test set" — set it to the horizon, and the training data stops where the real information would stop.

from sklearn.model_selection import TimeSeriesSplit

# Daily data, forecasting seven days ahead, thirty days tested per fold.
# gap drops the seven days that would not yet be known at forecast time.
tscv = TimeSeriesSplit(n_splits=5, gap=7, test_size=30)
for train_index, test_index in tscv.split(X):
    ...

Finally, the baselines, because they are what make a forecast defensible. Compute three before you model anything: repeat the last value, repeat the value one season ago, and take a moving average. A forecast that does not beat seasonal naive is not a forecast, and a surprising number of impressive-looking models do not. Report mean absolute error for a single series; when comparing across series of different sizes, scale the error by the naive baseline's error rather than reaching for a percentage, which explodes wherever the series touches zero. And report the error separately for each step of the horizon, because accuracy one day out and accuracy thirty days out are different numbers and averaging them hides which one you actually have.

What you should now be able to explain or do

Say why cross-validation that assumes independent samples is wrong here, in the guide's own terms. Identify trend and seasonality in a series and use differencing to remove them, stopping at the right point. Describe what the three ARIMA orders count and what the seasonal ones add. Say when a classical model is the better choice and when machine learning earns its cost. Build lag, rolling and calendar features that respect both the availability rule and the horizon rule. Set up a rolling-origin backtest with a gap equal to the horizon. Establish naive baselines and report error per horizon step.

Check yourself