Module 2 — Statistics Refresher for Model Builders

Part I · Foundations (Fast Track) · Priority ⭐ Recommended · Status: Draft v0.1 Prerequisites: Module 0 · Companion stack: numpy, pandas/polars, statsmodels, scikit-learn

Overview

Strategy research is applied statistics done under time pressure, on non-stationary data, with money attached. This module tightens exactly the statistical concepts your pipeline leans on daily — no derivations, no formulas-as-wallpaper. Every concept arrives as: plain-English intuition → runnable Python → a number you can check by hand.

The seven ideas covered here are load-bearing for everything downstream:

1. Returns arithmetic and annualization across timeframes
2. The true shape of returns: fat tails, skew, clustering
3. Why accuracy metrics lie on imbalanced financial labels
4. Stationarity — and why backtests quietly require it
5. Correlation vs cointegration (the entire foundation of Module 12)
6. Mean-reversion speed: the rubber-band half-life (Module 11)
7. The multiple-testing trap — the reason most published backtests die live (Module 19)

Returns: Log vs Simple, Compounding & Annualization

Simple return is the accounting view: profit divided by starting value. Log return is the bookkeeping view that makes sequences additive:

import numpy as np

price_0, price_1 = 100.0, 110.0
simple_return     = price_1 / price_0 - 1        # -> 0.10  (+10%)
log_return        = np.log(price_1 / price_0)    # -> 0.0953

# Additive across a sequence: chain of log returns sums to the total.
prices = np.array([100.0, 110.0, 99.0])
print(np.log(prices[1:] / prices[:-1]).sum())    # -> -0.01015
print(np.log(prices[-1] / prices[0]))            # -> -0.01015  identical

Meaning: simple returns compound multiplically ((1+r1)(1+r2)−1) and never average nicely across time; log returns add up, which is why models, Sharpe ratios, and HMM emissions all consume log returns. Convert back before reporting P&L to humans.

Annualization rescales a per-bar figure to “what a year of this would look like”. Volatility grows roughly with the square root of the number of bars, so each timeframe carries its own multiplier:

# Annualization factor = sqrt(bars per year). Assumptions matter!
BARS_PER_YEAR = {
    "D1 FX":      260,       # weekdays
    "H1 FX":      120 * 52,  # 24h x 5d week
    "M5 FX":      288 * 260, # 24h x 12 x 5d week
    "D1 crypto":  365,
    "M5 crypto":  288 * 365, # trades weekends
}
for k, n in BARS_PER_YEAR.items():
    print(f"{k:>10}: sqrt(n) = {np.sqrt(n):7.1f}")
# D1 FX: ~16.2 | H1 FX: ~79.0 | M5 FX: ~273.6 | D1 crypto: ~19.1 | ...

⚠️ Pitfall: Annualizing an M5 Sharpe by √74,880 magnifies any tiny per-bar edge into heroic yearly numbers — and magnifies estimation noise identically. An M5 strategy with per-bar Sharpe 0.01 prints “annualized 2.7”; the same estimate error at D1 prints 0.16. Compare strategies on per-trade or per-unit-time economics, not raw annualized Sharpe alone (Module 6).

The Shape of Returns: Fat Tails, Skew, Clustering

Textbook statistics assumes bell curves. Market returns break the bell curve in three repeatable ways:

• Fat tails — extreme days happen far more often than the bell curve allows.
• Skew — losses and wins have different shapes (crashes are sharper than rallies).
• Volatility clustering — turbulence arrives in clusters; calm weeks follow calm weeks.

from scipy import stats

# Normal curve reference: excess kurtosis = 0. Markets run far higher.
daily_returns = np.array([0.01, -0.002, 0.003, -0.05, 0.004, 0.002,
                          -0.003, 0.06, -0.001, 0.005])   # toy sample
print("excess kurtosis:", round(stats.kurtosis(daily_returns), 1))
# -> 2.9  (fat tails already visible in ten numbers)

# The famous scale of the problem: a -22% day under a bell curve.
sigma, mu = 0.01, 0.0002
z = (-0.22 - mu) / sigma
print("Oct-1987 was", round(z, 1), "sigmas ->",
      f"1-in-{1/stats.norm.sf(-z):,.0f} days")
# -> about -21 sigmas: impossible under normality. It happened.

Meaning: risk models built on bell curves underestimate crash frequency by orders of magnitude. Position sizing must assume tails exist (Module 23); drawdown expectations should be set by simulation, not by standard deviation alone.

Clustering shows up in any rolling-volatility chart — quiet months, then violent ones. That predictability-of-turbulence is itself tradable (Module 15) and is why GARCH-style forecasts earn their keep as model features (Module 18).

Why accuracy lies on imbalanced labels

Financial labels are usually lopsided: most candidate trades fail their barrier test. Accuracy rewards laziness there:

# Suppose 8% of Meta candidates are true WINs.
n_candidates   = 10_000
true_win_rate  = 0.08

# A useless model that predicts "LOSS" for everyone:
accuracy = 1 - true_win_rate                      # -> 0.92 (92%!)
precision_of_a_real_model = None                  # accuracy says nothing
# about catching the 800 winners. Use precision/recall or PR-AUC.

Meaning: a model scoring 92% accuracy may have found zero profitable trades. This is why your Meta tuning optimizes PR-AUC, not accuracy — now you know the statistics behind that choice.

Stationarity: The Quiet Prerequisite

A series is stationary when its long-run character doesn’t drift: same average level, same spread of movement, whenever you look. Prices themselves are almost never stationary (they wander); derived quantities — returns, z-scores, spreads between linked assets — often are.

Backtests implicitly demand stationarity: if the data’s behavior drifts over your sample, you’re fitting three different markets and reporting one blended fantasy.

from statsmodels.tsa.stattools import adfuller

def is_stationary(series: np.ndarray, name: str) -> bool:
    """Augmented Dickey-Fuller: H0 = 'series has a unit root (wanders)'."""
    stat, pvalue, *_ = adfuller(series, autolag="AIC")
    verdict = "stationary" if pvalue < 0.05 else "NON-stationary"
    print(f"{name:>14}: ADF p={pvalue:.4f} -> {verdict}")
    return pvalue < 0.05

rng = np.random.default_rng(42)
is_stationary(rng.normal(0, 1, 5000), "white noise")          # stationary
is_stationary(np.cumsum(rng.normal(0, 1, 5000)), "random walk")  # not

Practical translations:

• Random walk = today equals yesterday plus a coin flip: no memory of “fair value”, so nothing pulls it back. Raw prices behave this way.
• Autocorrelation = today’s value echoing yesterday’s. Positive echo → trends persist; negative echo → overshoots correct. Test with statsmodels.stats.diagnostic.acorr_ljungbox when in doubt.
• Volatility that changes over time (heteroskedasticity) — covered above as clustering; the term appears constantly in papers.

💡 Idea: You don’t need prices to be stationary — you need your trading quantities to be. Every feature you feed XGBoost should pass a stationarity sniff-test; a feature that’s really a wandering level will train beautifully in-sample and betray you out-of-sample. Your pipeline already stationarizes features (log-return-div-vol style); this is the reason why.

Correlation vs Cointegration: Two Drunks, One Leash

Correlation asks: do two series move together bar by bar? Cointegration asks something stronger: however far each wanders individually, does a stable long-run relationship hold the gap between them?

The classic picture: two drunks leaving a pub. Each stumbles randomly (neither path is predictable), but they hold a leash — the distance between them stays bounded even though both paths wander forever. Their steps correlate only loosely; their leash is what matters for pairs trading.

Why the distinction is not academic pedantry:

rng = np.random.default_rng(7)
n = 3000

# Two unrelated random walks — pure coincidence can correlate them highly.
a = np.cumsum(rng.normal(size=n))
b = np.cumsum(rng.normal(size=n))
print("corr(walk_a, walk_b) =", np.corrcoef(a, b)[0, 1].round(2))
# Frequently 0.80-0.95!  Correlation alone proves NOTHING here.

# Constructed truth: y is tied to x by a leash (plus small noise).
x = np.cumsum(rng.normal(size=n))
y = 0.7 * x + rng.normal(scale=0.5, size=n)
spread_leashed = y - 0.7 * x           # this IS the noise: bounded
spread_unleashed = b - a               # difference of two walks: wanders

is_stationary(spread_leashed, "leashed spread")    # ADF rejects -> cointegrated
is_stationary(spread_unleashed, "loose spread")    # ADF fails -> spurious pair

Meaning: correlation without cointegration is spurious regression — two drifting series look related because both drifted, and the “relationship” dissolves the moment you trade it. Pairs selection therefore runs a cointegration test (Engle–Granger via statsmodels.tsa.stattools.coint, or Johansen for baskets) on the spread, never a correlation rank alone. Full treatment lives in Module 12.

Mean-Reversion Speed: The Rubber Band & Half-Life

When a spread is cointegrated, the working question becomes: how fast does it snap back? The mental model is a rubber band stretched by a random tug each period — that process is called Ornstein–Uhlenbeck (OU), and it has one practical parameter: the half-life, the typical time for half of any dislocation to close.

No algebra needed — the half-life falls out of one lagged regression:

from numpy.linalg import lstsq

def half_life_bars(spread: np.ndarray) -> float:
    """Regress spread-changes on yesterday's spread level.
    Slope phi is negative when the band snaps back."""
    lagged  = spread[:-1]
    changes = np.diff(spread)
    phi = lstsq(lagged[:, None], changes[:, None], rcond=None)[0].item()
    assert phi < 0, "positive slope -> not mean-reverting"
    return -np.log(2) / np.log(1 + phi)

rng = np.random.default_rng(1)
ou = np.zeros(20000)
for t in range(1, len(ou)):                    # simulate a rubber band
    ou[t] = ou[t-1] * 0.95 + rng.normal(scale=0.2)
print("half-life:", round(half_life_bars(ou), 1), "bars")
# -> ~13.5 bars  (theory for pull=0.95: ln(0.5)/ln(0.95) ≈ 13.5)

Meaning: half-life converts directly into holding-period design. A pairs spread with a 60-bar half-life argues against a 5-bar scalp and a 500-bar swing alike — the trade should live near its own physics. The same estimate sizes z-score windows and stop horizons in Modules 11–12.

The Multiple-Testing Trap

Every backtest is a draw from a distribution of luck. Run one honest strategy: informative. Try four hundred parameter sets and keep the best: you’ve mostly measured luck — and luck this good does not follow you live.

Figure: how a pile of mediocre trials manufactures a superstar that dies in production.

flowchart TD
    pool[Pool of N trial configs<br/>most are pure noise]
    run[Run every trial<br/>on the SAME history]
    select[Select the best performer<br/>max of N noisy results]
    report{Reported as if<br/>one hypothesis?}
    inflated[Inflated OOS expectation<br/>guaranteed live disappointment]
    honest[Deflated expectations,<br/>honest selection]

    pool --> run
    run --> select
    select --> report
    report -->|"yes: survivor speaks"| inflated
    report -->|"no: all trials recorded"| honest

    classDef decision fill:#fff4d6,stroke:#d6a300
    classDef risk fill:#fde8e8,stroke:#c0392b
    classDef ok fill:#e6f4ea,stroke:#2e7d32
    classDef process fill:#f3f3f3,stroke:#888

    class report decision
    class inflated risk
    class honest ok
    class pool,run,select process

How to read this:

• The damage happens at the selection node: taking max-of-N and reporting it as if N were 1.
• The cure is not fewer trials — it’s bookkeeping: record all N, then discount the winner accordingly (the deflated-Sharpe mindset).
• This is precisely why your experiment ledger exists, and why Optuna trial counts belong in the final report.

Feel the trap numerically once and you’ll never forget it:

rng = np.random.default_rng(3)
years, bars_per_year = 2, 252

def noise_sharpe() -> float:
    rets = rng.normal(0, 1, years * bars_per_year)
    return rets.mean() / rets.std() * np.sqrt(bars_per_year)

trials = [noise_sharpe() for _ in range(200)]   # ALL pure noise!
best   = max(trials)
print(f"best of 200 zero-edge trials: Sharpe {best:.2f}")
# Typical output: 2.4-3.0  ->  a 'world-class' system from thin air.

Meaning: among 200 coin-flip strategies, someone scores Sharpe ≈ 3. If your research process tries hundreds of feature/target/barrier combinations (it does), your headline backtest is drawn from this distribution until proven otherwise. Defenses — purged cross-validation, combinatorial estimates, deflated Sharpe — are Module 19’s entire job.

Screening Features Without Fooling Yourself

Before any model runs, you want a cheap ranking of “which features know anything about the label”. Mutual information (MI) answers exactly that in plain words: how many bits of uncertainty about the label does knowing this feature remove? Zero bits = useless feature; higher = better.

from sklearn.feature_selection import mutual_info_classif

# features: (rows, k) already point-in-time; labels: triple-barrier outcomes.
mi_scores = mutual_info_classif(features, labels, discrete_features=False,
                                random_state=42)
ranking = sorted(zip(feature_names, mi_scores),
                 key=lambda kv: kv[1], reverse=True)
print(ranking[:8])

Two failure modes MI inherits from everything else in this module:

1. Drifting features inflate MI — a raw price level “predicts” labels because labels cluster in certain eras. Screen stationary transforms only (your pipeline’s log_return_div_vol family passes; raw levels fail).
2. MI says association, not causation or stability — a feature can score well on one regime and vanish in the next. MI ranks candidates; purged walk-forward decides survivors.

Testing It Honestly

Run this checklist before trusting anything built on the statistics above:

Hands-On Project

Deliverable: notebooks/stats_diagnostics.ipynb + a one-page findings memo. This is the curriculum’s first diagnostic instrument on your symbols.

Tasks:

1. Pull two visually correlated symbols from your MT5 universe (e.g., XAUUSD vs EURUSD, or BTCUSD vs ETHUSD) over ≥ 2 years of M5/D1 bars.
2. Report their correlation — then run the leash test: ADF on the raw spread, Engle–Granger via statsmodels.tsa.stattools.coint. Classify: genuinely linked, or spurious co-drift?
3. For whichever pair (or constructed spread) passes, estimate the OU half-life and translate it into a recommended holding window in bars and hours.
4. Demonstrate the multiple-testing trap on your own data: generate 100 randomized-label versions of one simple signal, plot the distribution of their Sharpe ratios, and mark where your real strategy sits.
5. Memo: three sentences per finding, written for your future self.

Acceptance criteria:

• Correlation AND cointegration reported side-by-side for the chosen pair.
• Half-life estimate includes a stability check (two subperiods agree within ~2×).
• The randomization histogram visibly shows best-of-N inflation.
• Every statistic states its bar timeframe and sample window.

Key Takeaways

• Log returns for computation, simple returns for reporting; annualization multipliers depend on bar frequency assumptions — write them down or comparisons become fiction.
• Returns have fat tails, skew, and volatility clusters; bell-curve risk numbers systematically understate crash frequency.
• On imbalanced financial labels, accuracy flatters any lazy model — precision-recall territory, which is why PR-AUC drives your Meta tuning.
• Backtests require stationary trading quantities, not stationary prices; ADF is the sixty-second sniff-test.
• Correlation is co-movement; cointegration is a leash. Pairs trading needs the leash — spurious co-drift dissolves the moment real money arrives.
• Half-life is the rubber-band speedometer: it converts spreads into holding periods directly.
• Best-of-N trials measure luck; with 200 zero-edge candidates, Sharpe ≈ 3 appears routinely — ledger everything, deflate everything, and let Module 19 mechanize the discipline.

References

• Ruey Tsay — Analysis of Financial Time Series, 3rd ed., ch. 1–3 (returns properties, stationarity, heteroskedasticity)
• Ernest Chan — Algorithmic Trading, ch. 2–3 (mean reversion, stationarity tests, half-life in practice)
• Granger & Newbold (1974) — Spurious regressions in econometrics — the original warning that correlation without cointegration is theater
• Marcos López de Prado — Advances in Financial Machine Learning, ch. 8 (multiple testing, deflated Sharpe, backtest overfitting)
• statsmodels — adfuller, coint documentation
• scikit-learn — mutual_info_classif documentation
• Next in sequence: Module 3 — Research Stack