Module 18 — Feature Engineering & Labeling for Financial ML

Part IV · Machine Learning · Priority 🎯 Core · Status: Draft v0.1 Prerequisites: Module 2, Module 9, Module 17

Overview

A model can only be as good as its raw materials. Features decide what the model is allowed to know — financially causal quantities, computed from information available at decision time — and labels decide what the model is trained to want: outcomes whose distribution matches real trading results, not an accounting fiction like “where will price sit in N bars.” Feed a strong learner sanitized inputs and fantasy targets, and you get a confident model that is precisely wrong in live trading.

The edge story of this module is asymmetry of honesty. Your counterpart — other algorithmic traders fitting ML models — overwhelmingly feeds raw indicator levels into fixed-horizon labels, validates with shuffled folds, and reports accuracy. Every one of those choices quietly inflates backtests. Causal features, path-aware labels, and uniqueness-weighted samples are cheap to build and compound into the gap between measured edge and real edge.

House position, stated bluntly: your pipeline already runs triple-barrier labels and a Primary→Meta meta-labeling architecture — this module is an audit-and-extend pass, not greenfield construction (Module 0’s capability map marks it accordingly). You will formalize why what you built works, then extend it: fractional differencing, sample-uniqueness weighting, vol-of-vol barrier sizing, and a standing leakage-hunt protocol.

🔧 For your pipeline: phase_4/meta_triple_barrier.py already implements route-specific profit-take/stop multiples and time barriers; the Primary-recall → Meta-precision split is meta-labeling. Treat Sections “Labeling Done Right” and “Meta-Labeling” as formalizations of running code, and spend your effort on the extensions catalogued near the end: uniqueness weighting (you currently don’t weight), vol-of-vol barriers, provenance audits, and Module 9 columns as new Meta context.

How It Works

Figure: two supply chains converge on training. Raw bars become causally transformed, stationarity-screened features; candidate signals become path-labeled outcomes weighted by uniqueness. Both feed purged folds (Module 19), which grade the calibrated Meta model that finally ships.

flowchart TD
    bars[(Raw M5 bars<br/>OHLCV + tick/spread)]
    causal[Causal transforms<br/>multi-horizon rets + vol ratios]
    htf[Module 9 composites<br/>htf_bias / mtf_struct]
    gate{Stationarity gate<br/>ADF passes?}
    frac[Fractional differencing<br/>or re-specify transform]
    store[(Point-in-time<br/>feature store)]
    cand[Primary candidates<br/>R0 / R1 / R2 signals]
    tbb[Triple-barrier labels<br/>vol-scaled pt/sl/time]
    uniq[Uniqueness weights<br/>inverse concurrency]
    folds[Purged walk-forward folds<br/>M19 machinery]
    sel[Feature selection<br/>MI + SHAP + clustering]
    model[Calibrated Meta model]

    bars --> causal --> gate
    htf --> gate
    gate -->|"yes"| store
    gate -->|"no"| frac
    frac --> gate
    store --> folds
    cand --> tbb --> uniq --> folds
    folds --> sel
    sel --> model

    classDef data fill:#e8f0fe,stroke:#4a86e8
    classDef process fill:#f3f3f3,stroke:#888
    classDef decision fill:#fff4d6,stroke:#d6a300
    classDef risk fill:#fde8e8,stroke:#c0392b
    classDef ok fill:#e6f4ea,stroke:#2e7d32

    class bars,store data
    class causal,htf,frac,tbb,uniq,folds,sel process
    class gate decision
    class model ok

How to read this:

• The left column is the feature ledger: every arrow adds information derived only from the past; the amber gate rejects anything still trending (a nonstationary series trains the model on era, not edge).
• The right column is the label ledger: signals become labeled episodes whose outcome depends on the path taken, and overlapping episodes get downweighted because they double-count the same price action.
• The convergence point is deliberate: features and labels meet only inside folds that respect time order — anything meeting earlier is a leak by definition.

Feature Families

Build features in families, each with a stated causality contract. A family is a recurring shape of question asked of price data; naming them keeps your schema reviewable and your leakage hunts targeted.

Multi-horizon returns and vol ratios are the workhorses. Markets repeat behavior at characteristic speeds — a 1-hour impulse and a 24-hour grind are different animals — so give the model several horizons plus their relative volatility, which normalizes regimes automatically:

# Multi-horizon returns + vol ratio: causal core of most Meta features.
import numpy as np
import pandas as pd

px = m5["close"]                                  # M5 closes, UTC index
for horizon in (12, 36, 96, 288):                 # 1h / 3h / 8h / 24h
    feats[f"logret_{horizon}"] = np.log(px / px.shift(horizon))

ret_5m = px.pct_change()
feats["vol_ratio_fast_slow"] = (
    ret_5m.rolling(36).std() / ret_5m.rolling(288).std())
# -> >1 means turbulence expanding vs daily baseline; regime-flavored signal

Multi-timeframe composites come free from Module 9: htf_bias, mtf_struct, and ltf_trigger already carry correct release times, so the Meta model inherits causal safety instead of having to re-earn it. Wire them in as-is; ablate later.

Stationarized indicators follow one law: the model may see differences, ratios, and z-scores — never raw levels. An RSI of 71 means nothing across eras; “RSI rose 6 points over the last hour while 20-day vol compressed” is comparable across years. Raw levels let the model memorize which year it is in — that is era-detection, not prediction.

Microstructure features are unique to broker data, which makes them your quietest competitive advantage: most public ML tutorials trade clean exchange data, while your MT5 feed exposes spread dynamics and tick flow nobody else’s model sees.

• Spread level and percentile: current spread in pips, plus its percentile within the same hour-of-day distribution (Module 4). Spread percentile doubles as a tradability gate.
• Gap statistics: frequency and size of gaps between bar opens — weekend gaps, rollover artifacts, news jumps. Gap-prone hours behave differently around stops.
• Tick intensity: MT5 tick volume per bar and its z-score. Activity surges precede both breakouts and exhaustion; the model learns which context disambiguates.

Cyclical calendar encodings map clocks onto circles so the geometry matches the physics — midnight is adjacent to 11pm, Friday is adjacent to Monday:

# Cyclical time encodings: hour 23 sits NEXT TO hour 0, not 23 steps away.
minutes_of_day = m5_index.hour * 60 + m5_index.minute
feats["hour_sin"] = np.sin(2.0 * np.pi * minutes_of_day / 1440.0)
feats["hour_cos"] = np.cos(2.0 * np.pi * minutes_of_day / 1440.0)

trading_dow = m5_index.dayofweek                  # trading week wraps Fri->Mon
feats["dow_sin"] = np.sin(2.0 * np.pi * trading_dow / 5.0)
feats["dow_cos"] = np.cos(2.0 * np.pi * trading_dow / 5.0)

Raw integers force a false ordering (Friday ≠ 5 steps from Monday); sin/cos pairs remove the cliff. Keep session flags (Asia/London/NY) alongside — trees split on flags cleanly, circles help smooth learners.

Correlation and regime features carry state, not just levels: forward-only HMM state probabilities from your Student-t layer, rolling XAUUSD↔BTCUSD correlation, and vol-of-vol (dispersion of realized-vol estimates). These are the columns that let the Meta model learn “my Primary works in transitional regimes and dies in expansion” — knowledge no single-bar indicator carries.

Stationarity Transforms

Models assume the training distribution resembles the live distribution. A series that wanders (price levels) violates that assumption: every statistic the model learns is secretly about a particular era. Module 2 introduced the ADF test as the referee; this section covers the three transforms that get features past it.

Rolling z-scores answer “how unusual is now versus recent now?” — subtract the trailing mean, divide by trailing std, and the output is unit-free by construction. Window choice is a modeling decision: short windows track fast dislocations, long windows encode slower positioning.

Vol-normalization divides moves by contemporaneous volatility (logret / rolling_vol, or ATR-scaled distances). A 40-pip move is enormous at 3am Asia and noise during London open; normalized, the model sees one comparable quantity across sessions.

Fractional differencing solves a genuine dilemma. Integer differencing (today minus yesterday) achieves stationarity but wipes out memory — and the predictive signal largely lives in slow-moving levels like cumulative momentum and persistent trends. Fully differenced price series look like white noise precisely because they are almost memory-free. Fractional differencing differences by a fraction d ∈ (0, 1): each transformed value becomes a weighted sum of all past levels with weights that decay toward zero, so partial memory survives while the series still passes ADF.

🧪 Evidence: López de Prado, Advances in Financial Machine Learning, ch. 5 shows that for many financial series a minimal d as low as 0.1–0.4 passes ADF while preserving correlation with the original (memory-rich) series — unlike d=1, which passes ADF but decorrelates nearly everything. Walkthrough: microalphas.com/fractional-differentiation.

The weights follow a simple recurrence — each successive weight shrinks geometrically-ish toward zero, so distant levels contribute ever less:

# Fractional-diff weights: how much of each PAST LEVEL survives in the
# transformed value. Magnitudes shrink toward zero => finite memory.
import numpy as np

def frac_diff_weights(d, k=10):
    w = [1.0]
    for i in range(1, k):
        w.append(-w[-1] * (d - i + 1) / i)   # binomial-series recurrence
    return np.array(w)

print(np.round(frac_diff_weights(d=0.35, k=10), 4))
# -> [ 1.     -0.35   -0.1138 -0.0626 -0.0414 -0.0303
#      -0.0234 -0.0189 -0.0157 -0.0134]

Meaning: with d=0.35 the newest level gets full weight, the previous level gets −0.35, and ten-bars-back contributes under 1.5%. The recipe: pick the minimal d that passes ADF (scan d from 0 upward, test, stop at the first pass), because smaller d retains more memory. Reserve it for your few slowest-moving series — price itself, cumulative-range proxies — where the memory dilemma actually bites; ordinary indicators transform fine with z-scores and ratios.

Labeling Done Right: The Triple-Barrier Method

Fixed-horizon labels ask “will price be higher in N bars?” and answer a question nobody trades. Real positions exit along the way — stopped out, take-profited, timed out — and two candidates with identical N-bar outcomes deserve different grades if one hit the stop en route. Training on endpoint-only labels teaches the model to ignore path dependency, then live trading charges you for the lesson.

The triple-barrier method labels each candidate by walking it forward through three exits:

1. Upper barrier — profit-take at entry + k·σ,
2. Lower barrier — stop-loss at entry − k·σ,
3. Vertical barrier — time limit after max_hold bars.

σ comes from a rolling volatility estimator (ATR or realized vol at entry time), so barriers widen in turbulent regimes and tighten in quiet ones: the same “+2σ target” means different absolute distances in London open versus Asia lunch, exactly matching how a vol-aware trader sizes exits. The label is whichever barrier touches FIRST; if neither does, the vertical barrier assigns a third class (timeout) — keep it, don’t silently drop it.

def triple_barrier_labels(close, sigma, pt_mult=2.0, sl_mult=2.0,
                          max_hold=48):
    """Label = FIRST barrier touched: +1 target, -1 stop, 0 time-out."""
    upper, lower = close + pt_mult * sigma, close - sl_mult * sigma
    labels = pd.Series(0, index=close.index, dtype=int)
    for t in range(len(close) - max_hold):
        win = close.iloc[t + 1:t + 1 + max_hold]
        up_hit = (win >= upper.iloc[t])
        dn_hit = (win <= lower.iloc[t])
        up_bar = up_hit.idxmax() if up_hit.any() else None
        dn_bar = dn_hit.idxmax() if dn_hit.any() else None
        if up_bar is None and dn_bar is None:
            continue                    # vertical barrier: stays 0
        if dn_bar is not None and (up_bar is None or dn_bar <= up_bar):
            labels.iloc[t] = -1         # stop first (tie = stop: pessimistic,
        else:                           #  same rule as Module 5 fills)
            labels.iloc[t] = +1
    return labels

This is the teaching skeleton — symmetric barriers, loop-based clarity. Your production version differs deliberately: route-specific pt/sl multiples and time limits per R0/R1/R2, which is correct because a mean-reversion fade and a breakout continuation have structurally different exit geometries.

The Overlap Problem and Sample Uniqueness

Barrier labels create overlapping episodes: a signal every few bars produces trades sharing the same stretch of price path, so when volatility expands, dozens of samples inherit essentially one outcome. The i.i.d. assumption behind standard training and scoring collapses — gradient updates double-count one market event, and worse, cross-validation leaks: a test-fold outcome sits inside a training-fold label horizon (Module 19 fixes the fold side with purging; this section fixes the sample side with weighting).

The remedy is a uniqueness weight per sample: the inverse of how many concurrent episodes covered each bar it lived through. An episode trading alone earns weight 1.0; one of ten simultaneous episodes earns roughly 0.1 — its outcome is mostly the crowd’s outcome, so it gets a correspondingly small vote.

# Uniqueness weights: overlapping trades share outcomes -> downweight crowds.
import numpy as np
import pandas as pd

episodes = pd.DataFrame({
    "t0": [0, 2, 4, 10],              # entry bar index
    "t1": [6, 8, 9, 14],              # exit bar (barrier resolution)
})

def uniqueness_weights(t0, t1):
    counts = np.zeros(int(t1.max()) + 1)          # concurrency clock
    for start, end in zip(t0, t1):
        counts[start:end + 1] += 1                # episode occupies these bars
    return np.array([
        (1.0 / counts[s:e + 1]).mean()            # avg inverse concurrency
        for s, e in zip(t0, t1)])

episodes["weight"] = uniqueness_weights(episodes["t0"], episodes["t1"])
print(episodes["weight"].round(2).tolist())  # -> [0.57, 0.43, 0.5, 1.0]

Hand-check episode A (bars 0–6): it runs alone on bars 0–1 (contribution 1 each), shares with B on bars 2–3 (½ each), shares with B+C on bars 4–6 (⅓ each) — averaging 4/7 ≈ 0.57. Episode D trades alone throughout: weight 1.0.

Pass the weights straight into the learner — every serious library accepts them, and your XGBoost Meta takes sample_weight natively:

# Convention: rescale so the MEAN weight is 1 (keeps regularization strength).
def normalize_mean_one(w):
    return w * (len(w) / w.sum())

model.fit(X_meta, y_meta,
          sample_weight=normalize_mean_one(uniqueness_weights(t0, t1)))

⚠️ Pitfall: unweighted overlapping labels inflate more than metrics — they distort the model. High-concurrency periods (news clusters, vol explosions) dominate gradients, so the Meta model overfits crowded regimes and underfits calm ones. Weighting is not cosmetic; it changes what gets learned.

Meta-Labeling: The Flagship Pattern

Meta-labeling splits the trading decision across two specialists. Stage 1 — the Primary — is a simple, interpretable rule set deciding SIDE (long/short) with high recall: fire often, catch nearly every real move, tolerate false positives. Stage 2 — the secondary ML model — looks at each Primary signal plus secondary features and predicts the probability that this specific signal succeeds (per the triple-barrier outcome). One probability, two uses:

• Filtering (veto): discard signals below a probability threshold.
• Sizing: map probability to bet fraction — high-confidence signals trade bigger.

The payoff is architectural: the Primary stays auditable (you can read its rules in a journal-entry sense — every entry has a documented reason), while the ML layer contributes precision without ever owning direction. Your R0/R1/R2 routing into route-local XGBoost Meta is this pattern, productionized.

Worked example — hand-checkable expectancy math. A rule-based system fires 100 signals with a 40% hit rate; winners pay +1.5R, losers cost −1R. Breakeven hit rate at payoff 1.5 is 1/(1+1.5) = 40%, so the raw system is exactly breakeven — before costs, a money-loser:

Same underlying market, same Primary logic. The Meta layer deleted the coin-flip cohort and doubled the survivor hit rate, converting −5R into +22.5R. Sizing extends the same number: survivors at p≈0.70 trade larger than p≈0.61 survivors, stacking a second benefit on top of filtering.

💡 Idea: think of meta-labeling as a loan committee. The Primary is the branch officer who forwards every plausible application; the ML underwriter prices each applicant’s default odds; the threshold decides who gets funding, and the probability sets the credit limit.

# Stage 2: success probability of PRIMARY signals, uniqueness-weighted.
meta_model.fit(
    X_secondary.loc[signal_mask],
    y_barrier_success.loc[signal_mask],
    sample_weight=normalize_mean_one(uniqueness_w.loc[signal_mask]),
)
p_success = pd.Series(
    meta_model.predict_proba(X_secondary)[:, 1], index=X_secondary.index)

veto = p_success < 0.60                                   # filtering use
bet_fraction = ((p_success - 0.50) * 2.0).clip(0.0, 1.0)  # sizing use

Calibrate before trusting either use (Module 17): uncalibrated tree probabilities make thresholds arbitrary and sizing maps nonlinear fiction.

Threshold Discipline

The veto threshold is where meta-labeling projects die politely. Raise it too far and the model becomes an all-veto champion: accuracy looks magnificent, PR-AUC flatters the résumé, and the book places zero trades. A filter that rejects everything is unfalsifiable and worthless — the metric that matters is not classification quality but net P&L after costs, evaluated out-of-sample.

Evaluate thresholds with a sweep table like this (illustrative numbers):

Two disciplines govern this table. First, judge rows by the P&L column only — hit-rate improvements that shrink net P&L are vanity. Second, notice that picking 0.60 because its row wins the sweep is itself a selection event: repeated across configurations, it is data snooping by another name. The sweep belongs inside purged, combinatorially validated evaluation — Module 19’s CPCV machinery exists precisely so this table’s winner survives being re-sliced.

Target Design

Before any labeling code runs, decide what quantity the model predicts — the choice shapes everything downstream.

Your stack is classification-first, and rightly so: barrier labels are discrete by construction, and route-local PR-AUC tuning assumes classes.

Horizon alignment rule: the label horizon must match the intended holding period. The classic trap — train on M5 labels, hold trades for a day — asks the model about the next four hours and deploys it on the next 24; the mismatch guarantees the learned probabilities mean nothing at deployment. Your pipeline holds this alignment by construction (M5 signals, M5-scaled barrier horizons); preserve it whenever adding systems, and re-derive barriers per timeframe rather than porting multipliers.

⚠️ Pitfall: grid-searching barrier multipliers (pt ∈ {1.5, 2, 3}, sl ∈ …, max_hold ∈ …) against final performance is hyperparameter snooping on the label itself. Every combination tried is a lottery ticket in Module 2’s multiple-testing trap. Either fix barriers from economic reasoning plus vol scaling, or run the grid inside CPCV so the trial count is paid honestly — Module 19.

Feature Selection Without Self-Deception

Feature selection’s real enemy is not dimensionality — it is self-deception: importance measures that flatter leaking or redundant columns. Use four tools, knowing each one’s bias.

Permutation importance shuffles one feature and measures the damage to validation performance. It is model-agnostic and directly answers “does this column earn its place?” — the closest thing selection has to an audit. Its blind spot: correlated features share credit, so permuted twins look individually weak.

SHAP values distribute each prediction’s credit back across the features — plain words: for every single trade, SHAP splits the gap between the prediction and the baseline among the columns that caused it, consistently. Aggregate SHAP over the validation set gives per-feature contribution with directional sign (did high values push toward WIN?). Its cost: computation, and the same correlated-credit-sharing caveat.

MDI caveats: Mean Decrease in Impurity (the default feature_importances_) is computed on training splits and is biased toward high-cardinality and high-variance features — it can promote noise columns that merely offer many split points. Treat MDI as a cheap first pass, never the verdict; confirm with permutation and SHAP on held-out folds.

Hierarchical clustering on feature correlation prunes redundancy structurally: cluster features whose pairwise correlations exceed a cutoff, keep one representative per cluster (highest MI score), and drop the rest. This attacks the credit-sharing problem at the source — fewer twins, cleaner attributions — before importance is even computed.

Screen first with mutual information (Module 2), which cheaply kills provably useless columns:

# MI screen: bits of label-uncertainty each feature removes (see Module 2).
from sklearn.feature_selection import mutual_info_classif

mi_scores = mutual_info_classif(X_train.fillna(0.0), y_train,
                                discrete_features=False, random_state=42)
mi_ranking = (pd.Series(mi_scores, index=X_train.columns, name="mi_bits")
              .sort_values(ascending=False))

dead = mi_ranking[mi_ranking == 0.0].index.tolist()
X_screened = X_train.drop(columns=dead)       # provably useless: zero bits
print(f"{len(dead)} dead features dropped; top-3 {mi_ranking.index[:3].tolist()}")

The leakage hunt protocol

One rule outranks every technique: a suspiciously strong feature is guilty until proven innocent. In genuinely noisy markets, no legitimate feature dominates; an outlier importance score is usually the feature reading the future through some seam. Protocol:

1. Recompute with shifted inputs. Rebuild the suspect feature using only data available one bar earlier (.shift(1) on every input). Legitimate predictive power degrades gracefully; leaked power collapses.
2. Audit the release timestamp. Trace every upstream join: resample label conventions, ffill boundaries, HTF bar completion (Module 9’s sync discipline). Off-by-one-bar joins are the most common leak.
3. Check survivorship-shaped strength. Columns describing “what happened later” (exit-related fields, post-event tags, anything derived from the label window) must never enter features.
4. Verify importance tables agree with your schema exclusions — see the provenance audit below.

# Leakage hunt demo: "dist_to_todays_high" posts MI 0.31 -- suspiciously hot.
suspect_raw = todays_high.reindex(m5.index, method="ffill") - m5["close"]
suspect_caution = suspect_raw.shift(1)     # yesterday's high ONLY: causal

auc_as_is   = purged_cv_auc(X.assign(dist=suspect_raw), y)
auc_shifted = purged_cv_auc(X.assign(dist=suspect_caution), y)
# auc 0.71 -> 0.52 after shifting: the "edge" WAS the look-ahead. Fix or drop.

A graceful decay (say 0.71 → 0.66) means the feature was real and merely lost freshness; a collapse to coin-flip territory is a confession.

For Your Pipeline: Audit-and-Extend

Four concrete extensions, ordered by expected effort-to-insight ratio. Each slots into m1-trading-model without touching the Phase-3 contract.

1. Sample-uniqueness weighting (do first — you currently don’t weight). Compute uniqueness_weights over your barrier episode intervals in Phase 4 and thread through fit(sample_weight=…). Expect the biggest effect in vol-expansion windows where R2 breakouts cluster; compare acceptance-selector t-stats before/after.
2. Barrier widths from vol-of-vol. meta_triple_barrier.py uses fixed route-local multiples. Test replacing static multiples with multiples scaled by a vol-of-vol factor (e.g., pt_mult × f(vol-of-vol percentile)) so barriers widen specifically when volatility itself is unstable — distinct from widening when volatility is merely high.
3. Provenance-exclusion audit. You already exclude candidate-provenance columns from Meta schemas. Verify the exclusion worked: run permutation importance and SHAP on a deliberately inclusive schema in a scratch study and confirm provenance ranks near zero. If it doesn’t, routing identity is leaking through a proxy column.
4. Module 9 composites as new Meta context. Feed htf_bias, mtf_struct, ltf_trigger (Module 9) into the route-local Meta datasets as context columns. Their release-time discipline is already proven; measure uplift per route with your nested walk-forward, then keep only what pays.

Run each extension through your experiment ledger with pre-registered hypotheses — the infrastructure for honest comparison already exists.

Testing It Honestly

Labelers and weighting functions are pure logic over synthetic data — fixture-test them before they touch production frames.

Labels must agree with economics. A strongly trending synthetic series (drift far above σ) should resolve mostly to WIN; a driftless chop series should resolve mostly to LOSS or timeout:

def test_trending_series_labels_mostly_win():
    rng = np.random.default_rng(11)
    trend = pd.Series(np.cumsum(rng.normal(0.10, 0.25, 600)))
    labels = triple_barrier_labels(trend, pd.Series(np.ones(600)))
    assert (labels == 1).mean() > 0.75       # drift >> sigma -> targets hit


def test_choppy_series_never_looks_trendy():
    rng = np.random.default_rng(12)
    chop = pd.Series(np.cumsum(rng.normal(0.00, 0.25, 600)))
    labels = triple_barrier_labels(chop, pd.Series(np.ones(600)))
    assert (labels == 1).mean() < 0.40       # no drift: wins cannot dominate


def test_uniqueness_weights_are_sane():
    w = uniqueness_weights(pd.Series([0, 2]), pd.Series([3, 5]))
    assert (w > 0).all()                      # strictly positive
    assert abs(normalize_mean_one(w).mean() - 1.0) < 1e-9   # scale convention

The leakage hunt deserves its own fixture: inject a deliberately leaking column (future high minus current close) into a scratch dataset, confirm the protocol catches it (MI spike, then collapse under .shift(1)), and wire the detection into CI so a regression that reintroduces the same seam fails loudly. A leak caught in a test costs nothing; the identical leak caught live costs a drawdown.

Finally, reconcile label statistics against trade reality: your labeled WIN rate should land in the same neighborhood as your Phase-5 backtest hit rate. A persistent gap between label-world and backtest-world is an accounting reconciliation problem — find the entry that doesn’t match its journal before trusting either.

Hands-On Project

Three deliverables, adapted to your pipeline vocabulary. Work them in a scratch branch; register each in the experiment ledger before running.

Project 1 — Triple-barrier relabel audit. Rebuild your historical pre-barrier fixed-horizon target (from ledger configs or reconstruct it), and run a side-by-side: old fixed-horizon labels vs current meta_triple_barrier.py labels. Train the route-local Meta on each, identical folds. Report label-distribution shifts, walk-forward stability deltas, and Phase-5 net-P&L realism deltas per route (R0/R1/R2).

Project 2 — Meta-labeling overlay on your best standalone rule-based system. Take the strongest rule system from Part III practice (e.g., a Donchian or session-breakout variant), generate its signals, and wrap them in a fresh Stage-2 meta-model with secondary features and uniqueness weights. Report hit-rate delta and net-Sharpe delta versus the unfiltered system, plus the full threshold sweep table.

Project 3 — SHAP leakage hunt. Run SHAP + permutation importance across your current feature set hunting for implausibly strong columns; apply the four-step protocol to each suspect; document at least one caught leak end-to-end (feature, mechanism, fix, before/after metrics). Include the provenance-exclusion verification from the pipeline section as part of the write-up.

Acceptance criteria:

• Project 1 compares both label schemes on identical folds with a per-route delta table (stability + net P&L).
• Project 2 reports hit-rate and net-Sharpe deltas plus a threshold sweep judged by post-cost P&L only.
• Project 3 documents ≥1 confirmed leak with mechanism, fix, and before/after AUC or P&L.
• Uniqueness weights implemented and threaded into fit(sample_weight=…) for at least one study.
• Fixture tests (trend/chop labels, weight sanity, injected-leak detection) pass in CI.
• Every configuration tried is logged with trial counts (feeding Module 19’s deflation math).

Key Takeaways

• Features determine the ceiling, labels determine what the ceiling means: causal, stationary inputs plus path-aware outcomes — anything less caps every model stacked on top.
• Never feed raw levels: differences, ratios, and z-scores travel across regimes; levels teach era-detection. Fractional differencing (minimal d passing ADF) rescues slow-moving series whose memory integer differencing would destroy.
• Triple-barrier labels grade the path, not the endpoint: vol-scaled barriers adapt to regime, FIRST-touch decides, and timeouts are a class you keep, not noise you drop.
• Overlapping episodes violate i.i.d.; uniqueness weights (inverse concurrency) restore one-outcome-one-vote — pass them to fit(sample_weight=…) today.
• Meta-labeling separates SIDE (interpretable Primary, high recall) from SUCCESS PROBABILITY (ML, precision), and one calibrated probability powers both veto-filtering and sizing.
• Judge thresholds by out-of-sample net P&L after costs, never accuracy or PR-AUC alone; an all-veto model is a perfect classifier and a useless trader.
• A suspiciously strong feature is a leak until proven otherwise: shift-rebuild, audit release timestamps, exclude label-window descendants, and make the hunt a permanent CI fixture.
• Barrier-multiplier grids and threshold sweeps are selections — pay for them inside CPCV (Module 19) or don’t trust the winner.

References

• Marcos López de Prado — Advances in Financial Machine Learning: ch. 3 (meta-labeling), ch. 4 (sample weights by uniqueness/return attribution), ch. 5 (fractional differentiation)
• Microalphas — Fractional Differentiation (walkthrough of the d-scan and memory retention)
• Stefan Jansen — Machine Learning for Trading, labeling & meta-labeling chapters (production-oriented implementations of triple-barrier workflows)
• scikit-learn — Permutation feature importance and the inspection guide (permutation importance, caveats)
• Internal: Module 2 — Statistics Refresher (ADF, MI screening) · Module 9 — Multi-Timeframe Design (composite columns) · Module 17 — ML Toolbox (calibration) · Module 5 — Backtesting (pessimistic fill rules shared by the labeler)
• Next in sequence: Module 19 — Validation Engineering (purged/CPCV folds that grade everything built here)