y[t] = trend + season + noise is a whole column of a spreadsheet with one extra rule: the rows are in order, and the split cannot be shuffled.
Shuffling days puts future labels next to past test points, so the model interpolates instead of forecasting. Walk-forward always trains earlier and tests later — the only split that survives deployment.
train ▸ past · test ▸ futureA direction, a calendar rhythm and randomness add up to the observed line. Name the parts and each gets the right treatment: difference the trend, lag the season, leave the residual alone.
y[t] = trend + season + noiseNaive copies the last value, seasonal naive copies the same slot a cycle ago, moving average smooths the last k, exponential smoothing fades the weights. Lose to these and the bug is yours.
MAE(model) < MAE(seasonal naive)A time series is data with a direction of travel: only the past may train, only the future may test, and the first model worth beating is yesterday’s value.
By the end you will be able to read a series into trend, season and noise, test it for stationarity and difference it, read an ACF to pick lags, build and race honest baselines, explain AR(1) and ARIMA, and evaluate forecasts with walk-forward validation and the right metric.
Standard ML promises that shuffling rows changes nothing. A time series breaks that promise twice — and the broken promise shows up as a validation score that is far too good.
Most ML assumes i.i.d. samples: independent of each other, drawn from one fixed distribution. Time series violates both halves. Not independent: today’s sales depend on yesterday’s, so each row carries information about its neighbours. Not identically distributed: December does not look like March, and a growing business does not look like a flat one. Shuffling rows destroys exactly the structure you are trying to predict.
The damage shows up at split time. A random 80/20 split can put March, May and July in training and April in test — so the model interpolates between labelled neighbours instead of forecasting. It predicts what already happened. The source for this lesson puts numbers on the gap: a model that scores 95% under random cross-validation can fall to 55% under honest time-based evaluation. The 40 points were never real.
The fix is a habit, not a trick: train only on the past, test only on the future, then slide the origin forward and repeat. That is walk-forward validation, and it is the most important idea in this lesson. It comes in two flavours: an expanding window keeps every historical day, while a sliding window keeps only the most recent stretch — reach for sliding when the world changes and old data hurts more than it helps.
| Split | Training set contains | Test set is | Score means |
|---|---|---|---|
| Random split | Past and future days mixed together | Future days sneak into the test set | Looks excellent; measures nothing a deployed model can do |
| Walk-forward | Only days before the test block | Test block is strictly in the future | Honest, noisier, and the number to trust |
The model is a 3-nearest-labeled-day lookup: for each test day it averages the three labelled days closest in time. The only difference is which days the shuffled row lets it see.
Every walk-forward fold trains only on days before its test block. The random split scatters future days through the training set, so each test day has a labelled neighbour near it — an advantage no deployed model ever gets.
The lab runs one model — a 3-nearest-labelled-day lookup — on one 120-point series, and changes only the split. Numbers from the default configuration (5 folds, 48 training days minimum):
The model did not get smarter between the two rows. The shuffled split simply handed it information — labels from the future — that a deployed model never receives. When a shuffled score beats the walk-forward score, suspect the split before celebrating the model.
You shuffle a daily sales series, run 5-fold CV, and get a beautiful score. Three weeks later the deployed model is much worse. What went wrong?
Before modeling, split what you see into a direction, a rhythm, a slow wander and randomness. Each part is extended differently — and some parts should not be extended at all.
Plot the raw series first; thirty seconds of looking beats an hour of automated guessing. Then name the parts: a trend you could lay a ruler along, a seasonal rhythm that repeats on a calendar clock, a cycle that rises and falls on no fixed schedule, and the residual that is left over. The standard teaching model is additive: y[t] = trend[t] + season[t] + noise[t].
Real business series are often multiplicative: a December spike is +20% of whatever the current level happens to be, not +20 units. The fix is one column of arithmetic: take logs and the product becomes a sum, log y = log trend + log season + log noise. Forecast in log space, then exponentiate to return to the original units.
| Component | What it is | Where you meet it |
|---|---|---|
| Trend | The long-term direction: up, down or flat. | Revenue growing 10% a year; global temperature rising. |
| Seasonality | A repeating pattern tied to a fixed calendar period. | Retail spikes every December; air conditioning peaks every July. |
| Cycle | A rise and fall with no fixed period — longer and looser than a season. | Housing markets, business cycles, sunspot activity. |
| Noise (residual) | What is left after the other three are removed. | If it looks like white noise, the decomposition captured the signal. |
Turn trend, seasonality and noise up and down and watch the observed series become the sum of its components. Switch to the difference view to remove the trend.
Turning the trend to zero is what differencing does for you: the rolling mean of the difference stays flat even while the original climbs.
Take y = [13, 8, 9, 15, 10, 11] and a season length of 3. The classic moving-average recipe estimates the seasonal shape first.
Now the multiplicative version. A shop sells 100 units in a normal month and 120 in December — a 20% spike, +20 units. Next year the shop has grown: a normal month is 200 units and December is 240, a +40-unit spike. The additive seasonal effect grew; the ratio did not. In logs it is constant: ln(120) − ln(100) = ln(1.2) = 0.182 and ln(240) − ln(200) = ln(1.2) = 0.182. When the size of the swing scales with the level, log-transform first and everything you know about additive decomposition applies again.
Most forecasting methods assume the rules of the series do not change. When the mean drifts, the model trained on January is simply wrong about February — unless you difference.
A series is stationary when its statistical properties — mean, variance and autocorrelation structure — do not change over time. Stationary does not mean flat: a series can jump wildly and still be stationary, as long as the size of the jumps is stable and it keeps returning to the same level. What breaks stationarity is a rule that moves: a trending mean, growing variance, or seasonality that changes shape.
The practical check is to compare rolling statistics. Compute the trailing mean and standard deviation over a window and watch them: if the rolling mean drifts, the series is non-stationary. The lesson source also compares the two halves of the series — flag it when the half-means differ by more than half the overall standard deviation, or when the larger half-variance is more than twice the smaller.
The standard fix is differencing: model the change, diff[t] = y[t] − y[t−1], instead of the level. Each round removes one degree of trend. If one round still drifts, apply another; most real series need at most two. (The formal statistical test is the Augmented Dickey–Fuller test, where p < 0.05 rejects non-stationarity; this lesson sticks to the visual check.)
Read the variance column: it is not that differencing makes a series “smaller” — it makes the rule constant. Each round lowers the polynomial degree of the trend by one: a line needs one round, a parabola two, a cubic three. For a stationary series, differencing is not free: white noise with variance 1 becomes first differences with variance 2, because subtracting two independent draws adds their variances. Over-differencing buys a constant mean and pays with extra noise.
After one round of differencing, the rolling mean is finally flat — but the rolling standard deviation is growing every year. What is the right next move?
Correlate a series with a shifted copy of itself and you get its memory. The lags that stick out become the features; the lags that vanish become noise you refuse to feed the model.
Autocorrelation is the correlation between a series and a lagged copy of itself. Pick a lag k, pair every day with the day k steps later, and measure how well the pairs move together. The autocorrelation function (ACF) is that number for every k. In plain English: multiply each value by the value k steps later, average, and divide by the variance of the series.
The same thing drawn as a scatter is a lag plot: x is today, y is tomorrow, one dot per day. A cloud stretched along the diagonal means strong positive autocorrelation — tomorrow looks like today. That cloud’s fitted slope is exactly ρ(1).
Three readings matter. A slow decay says the series remembers far back. A spike at lag 7 on daily data (or lag 12 on monthly data) says seasonality: the value repeats with the calendar. And bars that sit inside the dashed band — roughly ±1.96/√n, so ±0.14 at n = 200 — are indistinguishable from noise. The partial autocorrelation (PACF) sharpens the picture by removing indirect correlations: if lag 3 matters only because lag 1 does, ACF still shows it but PACF cuts it off.
Simulate an AR(1) with memory φ. Left: each point pairs a day with the next. Right: how strongly the series correlates with its own past, with the theoretical φᵏ dotted on top.
φ is both the lag-1 autocorrelation and the decay rate: the ACF of an AR(1) is φᵏ, so each extra lag multiplies the memory by φ again.
Take the tiny series y = [1, 2, 3, 4]. Its mean is 2.5, so the deviations are [−1.5, −0.5, 0.5, 1.5] and the denominator is the sum of their squares: 5.0.
For the AR(1) model — tomorrow is a fraction φ of yesterday’s deviation plus fresh noise — the ACF has a closed form: ρ(k) = φᵏ. With φ = 0.8 the bars fall 0.80 → 0.64 → 0.51 → 0.41: each lag multiplies the memory by φ. The deviation halves after ln(0.5)/ln(0.8) = 3.1 steps, and it drops below the n = 200 noise band after about 9 lags — which is exactly how many lags a model should bother with.
A daily series shows ACF bars inside the noise band for lags 1–6, a clear spike at lag 7, and a smaller spike at lag 14. What is the most useful feature set?
A forecast is only useful if it beats the laziest reasonable guess. Four one-line baselines set that bar, and one of them is embarrassing to lose to.
The lesson source is blunt about it: establish baselines before building any model, because if your fancy ML model loses to the seasonal naive, you have a bug — most often future leakage in the features, a broken evaluation split, or a series that is genuinely unpredictable. Baselines are also the honest denominator for every later improvement claim: “8% better MAE” means little until you know what last week’s value scores.
Naive (persistence) predicts tomorrow equals today. Seasonal naive predicts the same point one full season ago — last week’s Tuesday for a daily series. Moving average predicts the mean of the last k values, smoothing noise but lagging turns. Exponential smoothing is the same idea with weights that fade geometrically: S[t] = α·y[t] + (1−α)·S[t−1], so the newest value gets weight α, the one before it α(1−α), and so on.
| Baseline | Rule | When it is hard to beat |
|---|---|---|
| Naive / persistence | Tomorrow = today | Stable or slow-moving series with no calendar shape |
| Seasonal naive | Today = same point one season ago | Anything with hourly, weekly or yearly rhythm |
| Moving average | Forecast = mean of the last k values | Noisy series with a stable level; k is a knob |
| Exponential smoothing | Weighted mean with fading weights | Same as moving average, but recent points count more |
Every method predicts the same held-out last 12 points. The table is computed from the drawn series — change the knobs and watch the standings move.
The seasonal naive copies what happened one full cycle ago. When a series breathes on a calendar, it is a shockingly strong baseline — which is exactly why it belongs in every race.
| Method | MAE | RMSE | MAPE |
|---|---|---|---|
| naive | 6.74 | 7.99 | 6.2% |
| seasonal naive ✓ | 2.66 | 2.97 | 2.5% |
| moving average | 7.67 | 9.02 | 7.1% |
| exp. smoothing | 6.87 | 8.13 | 6.4% |
Take y = [10, 16, 12, 18, 14, 20], a series with a 3-step calendar rhythm, and hold out the next value, 19. Each baseline forecasts one number except the seasonal one, which has a shape to copy.
On this toy the two copies win and the two smoothers trail, because the series alternates and the smoothers average the highs with the lows. That is the point of a race: it is cheap, it is honest, and it tells you which kind of structure — memory or level — actually lives in the series. The real race in the lab is run on 96 training points and scored on 12 held-out ones, with every number computed from the displayed data.
The classical model has three letters. Take them one at a time and ARIMA stops being a black box: past values, past mistakes, and the differences that made the series stable.
AR stands for autoregressive: predict from the past values themselves. AR(p) is plain linear regression whose inputs are the last p values of the same series. The smallest version, AR(1), is worth memorizing:
Read it as a sentence: today sits μ, plus φ of yesterday’s gap from μ, plus news. If |φ| < 1 the gap shrinks by a factor φ each step — mean reversion — and the process is stationary with variance σ²/(1−φ²). At φ = 1 the pull disappears and you have a random walk: the mean wanders off and the variance grows without bound. Anything above 1 diverges faster still. This single number is why the lesson hammers stationarity: φ is only meaningful when it is below 1.
MA stands for moving average of errors — correct today’s forecast using the mistakes of the recent past, which is not the same as the moving-average baseline. I is for integrated: apply the differencing from the previous chapter d times before fitting. ARIMA(p, d, q) is therefore “past values, d differences, past errors”. For most practical problems, lag features plus gradient boosting remain the strongest starting point — it takes external features naturally, needs no stationarity, and is easier to debug.
| Approach | Best for | Seasonality | External features |
|---|---|---|---|
| Lag features + ML | Tabular problems with external features | Add calendar columns | Yes |
| ARIMA | One univariate series, short horizons | SARIMA variant | Limited (ARIMAX) |
| Exponential smoothing | Simple trend and seasonality | Yes (Holt–Winters) | No |
| Forests / gradient boosting | Many features, many series, messy data | With calendar features | Yes |
Each path obeys y[t] = 50 + φ(y[t−1] − 50) + noise. The thick curve is the same rule with the noise switched off: watch the gap to 50 shrink by a factor of φ each step.
With |φ| < 1 the process has a home to return to; the forecast is a shrinking fraction of the current gap. Raise φ toward 1 and both the memory and the error bars stretch out.
Take μ = 50, φ = 0.8, noise σ = 2, and suppose the last observed value is 55.00. The gap is +5, and the forecast for h steps ahead is μ + φʰ·(55 − μ).
Two different kinds of “settling” are in that block. The centre line converges to the mean at rate φ; the spread expands toward the stationary variance σ²/(1−φ²) at rate φ². A forecast interval that widens with the horizon is not hedging — it is the arithmetic of compounding uncertainty.
An AR(1) fit on a raw, untransformed series reports φ = 1.02. What does the model imply about the future?
Standard models need a table. One time series becomes a table the moment you shift it — and that one shift is where the most common bug in forecasting lives.
Take the source’s tiny series [10, 12, 14, 13, 15] and build lag-1 and lag-2 columns. Each row predicts today from the values strictly before it:
| lag_2 | lag_1 | target y[t] |
|---|---|---|
| 10 | 12 | 14 |
| 12 | 14 | 13 |
| 14 | 13 | 15 |
The first two rows drop out because they lack a full history, and now any regressor — linear regression, random forest, gradient boosting — can consume the table. Richer columns come from the same move: rolling mean/std/min/max over windows, calendar columns (day of week, month, holiday), differenced values, expanding statistics, ratios like current ÷ rolling mean, and interactions such as lag_1 × weekend. Use the ACF to choose how many lags: include the seasonal ones, and stop where the bars stop. More history is not free; every lag is another parameter that can fit noise.
The target alignment trap. Every feature must be known at prediction time. If the value at time t sneaks into the inputs, you have a perfect predictor and a completely useless model. The lag-builder lab below lets you flip that switch and watch every score become a lie.
One column of numbers is not a training table — but its past is. Each usable row predicts today from yesterday and the days before. Flip the leak switch to see the bug every beginner ships once.
| t | y[t] | lag_1 | lag_2 | roll_3 | target |
|---|---|---|---|---|---|
| 0 | 10 | — | — | — | 10 |
| 1 | 12 | 10 | — | — | 12 |
| 2 | 14 | 12 | 10 | — | 14 |
| 3 | 13 | 14 | 12 | 12.0 | 13 |
| 4 | 15 | 13 | 14 | 13.0 | 15 |
| 5 | 17 | 15 | 13 | 14.0 | 17 |
| 6 | 16 | 17 | 15 | 15.0 | 16 |
| 7 | 18 | 16 | 17 | 16.0 | 18 |
| 8 | 20 | 18 | 16 | 17.0 | 20 |
| 9 | 19 | 20 | 18 | 18.0 | 19 |
Rows are still in time order. Shuffling this table is the next trap: it would scatter future rows into the training set.
Here is the same idea in the source lesson’s Python, plus the walk-forward splitter it pairs with. Two functions, no library magic: shift the column, drop the incomplete rows, and always yield a test block that starts where the training data ends.
def make_lag_features(series, n_lags):
"""row t = [y[t-1], ..., y[t-n_lags]] -> y[t]"""
n = len(series)
X = np.full((n, n_lags), np.nan)
for lag in range(1, n_lags + 1):
X[lag:, lag - 1] = series[:-lag] # shift the column
valid = ~np.isnan(X).any(axis=1) # drop the first n_lags rows
return X[valid], series[valid]
def walk_forward_split(n_samples, n_splits=5, min_train=50):
"""expanding train window, next block always the test set"""
step = max(1, (n_samples - min_train) // n_splits)
for i in range(n_splits):
train_end = min_train + i * step
test_end = min(train_end + step, n_samples)
if train_end >= n_samples:
break
yield slice(0, train_end), slice(train_end, test_end)
X, y = make_lag_features(series, n_lags=7) # weekly rhythm -> 7 lags
for train_idx, test_idx in walk_forward_split(len(X)):
model = Ridge(alpha=1.0).fit(X[train_idx], y[train_idx])
score = mean_absolute_error(y[test_idx], model.predict(X[test_idx]))Forecasts are regression errors with time-aware context. Take four predictions and their labels, first a well-behaved set:
Choose the metric before the model. MAE speaks in original units and treats all misses equally. RMSE punishes one large error more, which is what you want when stock-outs or outages are expensive. MAPE is scale-free — useful across series of different sizes, but it explodes near zero and it treats an over-forecast and an under-forecast of the same size differently, so say which direction your loss really pays for. Pinball loss is the metric for prediction intervals: it charges 0.9 per unit for under-forecasting a 90% quantile and 0.1 for over-forecasting, so the minimizer is the quantile you asked for.
| Horizon strategy | How it works | Strength | Cost |
|---|---|---|---|
| Recursive | Forecast 1 step, feed the prediction back, repeat | Simple, one model | Errors compound with horizon |
| Direct | Train one model per horizon (a t+1 model, a t+5 model) | No compounding; per-horizon tuning | Fewer rows per model; no shared information |
| Multi-output | One model predicts every horizon at once | Shares information | Needs a multi-output model or custom loss |
Answer before you look. The split question and the lag question are the two that decide whether a production forecast is honest.
0 / 5 answered · 0 correct
01Why is a random train/test split invalid for time series data?
02What does it mean for a time series to be stationary?
03What is the purpose of differencing a time series?
04Lag features convert a time series into a supervised learning problem. What is a lag-3 feature for predicting y[t]?
05Walk-forward validation splits time data into expanding or sliding windows. Why is this better than K-fold CV for time series?
Click a card to swap the lazy description for what it actually means.
Four problems with numbers — differencing, lag selection, the split gap, and a multi-step error cone. Try first; a worked answer is one click away.
For y[t] = 5t + 3 over t = 0…4 the series is [3, 8, 13, 18, 23]. The first-half mean is 5.5, the second-half mean is 18 — a shift of 12.5, well above the source code's threshold of 0.5·std = 0.5 × 7.07 = 3.54 — so it is flagged non-stationary. The first difference is [5, 5, 5, 5]: constant, zero variance, stationary after one round. For a quadratic y[t] = t² = [1, 4, 9, 16, 25], the first difference [3, 5, 7, 9] still drifts and the second difference [2, 2, 2] is flat: two rounds. Each difference lowers the polynomial degree by one, so a degree-d trend needs d rounds; real series almost never need more than two.
With n = 200 the 95% significance band is ±1.96/√200 = ±0.139. On a weekly series the ACF typically spikes at lag 7 (about 0.8) and lag 14 (about 0.6) while lags 1–6 sit under the band. A model using just lag 7 and lag 14 carries 3 parameters (two weights plus a level) and captures the whole repeating shape; a model using lags 1–7 carries 8 parameters and spends five of them fitting noise, so its validation error is usually worse. Rule: take the lags whose bars poke outside the band, always include the seasonal lags, and stop before the feature count starts to rival the row count.
With the default 5 folds and a 48-day minimum training window, the walk-forward folds score MAE 3.19, 1.63, 1.62, 1.71, 1.92 — mean 2.01. The same model under a shuffled 80/20 split scores MAE 1.28, about 36% lower. Nothing changed about the model: the shuffled split left training days on both sides of nearly every test day, so the three nearest labelled neighbours are one step away in time. Walk-forward only ever looks backward, and its test days sit up to 14 steps beyond the last labelled day. The 0.73 MAE gap is the leak measured — treat a shuffled score this side of walk-forward as a red flag, not a result.
Build the rolling columns from lag values only: series.rolling(7).mean() at row t includes y[t] and leaks the target, so shift it by one or compute it from the lag columns. For ψ = 0.8 and σ² = 4, the recursive forecast-error variance σ²·Σφ²ʲ grows 4.00 → 6.56 → 8.20 → 9.25 → 9.98 over horizons 1–5 and plateaus at the stationary variance σ²/(1−φ²) = 11.11, a standard deviation of 3.33. That is the cone a recursive forecast must draw around itself. Direct forecasting trains one model per horizon: the horizon-5 model learns from rows where y[t+5] is already known, so errors do not compound, but each model sees fewer rows and the horizons cannot share information. Recursive for short horizons, direct or multi-output beyond.
You do not need to master these here. Each one gets a proper treatment in its own lesson; the one-line meaning is enough to keep reading. Orange dotted underlines in the prose point back to this list.
This lesson is a port of an open course. Everything here traces back to it — and the next step is running the code yourself.
Lesson text adapted from AI Engineering from Scratch (Phase 02, Lesson 15) and the Math Foundations Notebook reference build. Interactive figures, the six labs, the hand-worked numeric examples and worked exercise answers are original to this page. Every forecast and error a lab prints is computed live from the series it displays.