Module 3 — Research Stack & Where Your Pipeline Fits

Part I · Foundations (Fast Track) · Priority ⭐ Recommended · Status: Draft v0.1 Prerequisites: Module 0, Module 2

Overview

Strategy code is easy; research infrastructure is what separates a pile of notebooks from a repeatable edge factory. This module defines the reference anatomy of a professional quant research workflow — the stages, the contracts between them, and the reproducibility habits that make results trustworthy months later.

Then it does something most curricula skip: it audits your actual pipeline against that anatomy. You run m1-trading-model, which already implements large parts of this reference design (content-addressed cache DAG, pre-registered ledger, atomic model bundles). The goal here is not admiration — it’s producing a written gap list that feeds Modules 17–19.

The Reference Research Pipeline

Figure: the anatomy of a professional research stack. Every arrow is a contract; every box owes its neighbors a stable schema. The dashed feedback loop is where most retail stacks are missing limbs.

flowchart TD
    ingest[(1 Data ingestion<br/>point-in-time)]
    features[2 Feature engineering<br/>causal transforms only]

    ingest --> features
    features --> label[3 Labeling<br/>triple-barrier etc.]
    label --> train[4 Train / validate<br/>purged walk-forward]
    train --> evaluate[5 Evaluation harness<br/>costs, tearsheets, ablations]
    evaluate --> registry[6 Model registry<br/>hashed artifacts]
    registry --> deploy[7 Deployment<br/>live execution]
    deploy -.->|"drift signals,<br/>live-vs-backtest gaps"| monitor[8 Monitoring]
    monitor -.->|"retrain / retire triggers"| train

    classDef data fill:#e8f0fe,stroke:#4a86e8
    classDef process fill:#f3f3f3,stroke:#888
    classDef ok fill:#e6f4ea,stroke:#2e7d32
    classDef risk fill:#fde8e8,stroke:#c0392b

    class ingest data
    class features,label,train,evaluate,monitor process
    class registry ok
    class deploy risk

How to read this:

• Stages 1–5 form the research half (fast iteration); 6–8 form the operations half (money at risk). The registry is the airlock between them: nothing reaches deployment except through a hashed, documented artifact.
• The red node is deliberate: deployment is where mistakes become expensive, so entry into it must be gated and boring.
• The feedback loop (monitoring → training) is the difference between a strategy that decays quietly and one you notice decaying (Module 19).

Stage contracts

Meaning: the invariants are the architecture. Schemas can evolve; the invariants (causality, purge discipline, parity, immutability) may not be violated even once without invalidating every downstream result.

Two Ways to Simulate Time

Research code simulates markets in one of two styles. Knowing which you’re in — and what each silently assumes — prevents an entire genre of bugs.

Vectorized treats history as one big array: signals computed column-wise, positions implied by shifts. Fast, ideal for screening hundreds of variants.

import pandas as pd

df = pd.DataFrame({"close": [10, 10.4, 10.2, 10.9, 11.2, 10.8]})

# Signal known at close of bar t -> tradable at bar t+1. Shift FIRST.
zscore = (df["close"] - df["close"].rolling(20).mean()) / df["close"].rolling(20).std()
position = np.sign(zscore).shift(1).fillna(0.0)     # <- the guard

bar_pnl = position * df["close"].diff()             # vectorized P&L

⚠️ Pitfall: The .shift(1) is the entire honesty of the snippet above. A vectorized signal without it “decides” using the close and profits off that same close — look-ahead bias so common it has its own name in every backtesting post ever written.

Event-driven walks bar-by-bar (or tick-by-tick) through a queue: orders, fills, latency, partial executions modeled explicitly. Slower, but it’s where realistic fills and portfolio state live.

Engines you’ll meet in this curriculum:

Module 5 reconciles all of them on one system — including where the Strategy Tester disagrees with your Python stack, and why those disagreements are the lesson.

Reproducibility Engineering

A result you cannot re-run six months later never happened. Four habits make research reproducible:

1. Config-as-code

Every experiment parameter lives in a file, not in notebook-cell history:

import tomllib
from dataclasses import dataclass

@dataclass(frozen=True)
class ExperimentConfig:
    symbol: str
    timeframe: str
    barrier_pt_atr: float
    barrier_sl_atr: float
    max_hold_bars: int
    fold_scheme: str            # e.g. "purged_kfold_v1"

with open("experiments/R042.toml", "rb") as fh:
    cfg = ExperimentConfig(**tomllib.load(fh)["experiment"])

2. Seed everything

def seed_everything(seed: int) -> None:
    import random
    random.seed(seed)
    np.random.seed(seed)
    # torch.manual_seed(seed)          # when neural nets join (Module 20)

SEED = 42                              # one constant, referenced everywhere
seed_everything(SEED)

3. Snapshot the data

Record exactly which data window produced a result — ideally by content hash, not by date string:

import hashlib, json

def fingerprint(frame_bytes: bytes) -> str:
    """Short content hash identifying a dataset snapshot."""
    return hashlib.sha256(frame_bytes).hexdigest()[:12]

print(json.dumps({
    "dataset_sha": fingerprint(open("data/xauusd_m5.parquet", "rb").read()),
    "rows": 1_482_113,
    "window": ["2017-01-01", "2026-05-13"],
}))

4. Register outcomes, not vibes

Append an evaluation record next to every pre-registered card (Module 0’s loop):

record = {
    "id": "R042",
    "status": "completed",
    "config_sha": fingerprint(cfg.__repr__().encode()),
    "oos_net_sharpe": 1.31,
    "n_oos_trades": 214,
    "verdict": "promote-to-paper",   # -> reject / hold
}
with open("experiment_ledger.jsonl", "a") as f:
    f.write(json.dumps(record) + "\n")

Notebook vs Script Boundary

House rule, stated once: notebooks explore; scripts ship.

• A notebook may be messy, stateful, and half-wrong — that’s what exploration is.
• The moment a finding graduates into the pipeline (a feature, a filter, a validation step), it becomes an imported function with tests. Copy-pasting notebook cells into production modules is how silent divergence begins.
• Promotion ritual: extract function → add unit test → delete the cell → import the module in the notebook. The notebook then proves the shipped code reproduces the exploratory finding.

This mirrors how your production repo already works (src/lgbm_pipeline modules with pytest coverage; notebooks as private scratch) — here it’s codified so research repos converge on the same shape.

Audit: Mapping Your Pipeline to the Anatomy

🔧 For your pipeline: m1-trading-model already implements this reference design further than most professional shops:

Anatomy stage	Your implementation	Verdict
Point-in-time ingestion	Phase 1 + exclusive as_of boundary	✅
Causal features	Phase 2 + content-addressed cache DAG (polars_v15)	✅ exemplary
Honest labels	meta_triple_barrier.py as sole supervised path	✅
Purged validation	Nested walk-forward + untouched holdout, v12 tuning	✅
Evaluation harness	Phase 5 engines + canonical diagnostics	✅
Registry	Atomic hashed final_model_bundle_v2	✅
Deployment	src/live supervisor (journal-authoritative)	✅ hardened
Monitoring feedback	Schema-v6 snapshots exist; drift consumers absent	❌ gap

Verified gaps carried into later modules: drift monitors & parity harness (M24), combinatorial/PBO statistics (M19), model zoo beyond XGBoost (M17), sequence models (M20), session-granular cost curves (M22).

That table is the template for your own audit. Most retail pipelines fail at stages 1 (survivorship, restated fundamentals), 4 (shuffled CV), and 8 (no monitoring at all); yours fails only at 8 — which is precisely why Module 19’s drift work and Module 24’s parity harness sit on the critical path of your study plan.

Testing It Honestly

An infrastructure audit checklist — answer per stage, in writing:

1. Ingestion: Can I reproduce any past result’s exact input window? Who else could have modified it?
2. Features: Is there a test asserting no feature uses information after bar close?
3. Labeling: Could two researchers, given the config, generate byte-identical labels?
4. Train/validate: Does any selection decision touch the holdout? Show me the ledger.
5. Evaluation: Is the cost model identical to (or conservative vs) live fills?
6. Registry: Can I prove which data/config/code produced the deployed model?
7. Deployment: If I feed the same historical bar to training code and live code, do outputs match bit-for-bit?
8. Monitoring: What alarm fires when live diverges from expectation — and who hears it?

Any “don’t know” is a finding. Findings go straight into the project below.

Hands-On Project

Deliverable: docs/research/pipeline_audit.md — your stack scored against the reference anatomy, plus a ranked gap backlog.

Tasks:

1. Build the eight-row stage table (anatomy stage → your implementation → verdict ✅/⚠️/❌) for your own research setup, not just the production repo. Research-side gaps usually outnumber production ones.
2. Answer the eight audit questions above with evidence (file paths, test names, ledger entries) — not intentions.
3. Write a top-5 gap backlog, each item linked to the curriculum module that closes it and a rough effort guess.
4. Run the promotion ritual once: take any useful notebook-only function you have, ship it behind a test, and record the before/after.

Acceptance criteria:

• Table covers all eight stages with evidence links, not adjectives.
• At least three findings marked ⚠️ or worse, each with a named module owner.
• Gap backlog ordered by leverage (what unblocks the most downstream study).
• One function demonstrably promoted from notebook to tested script.

Key Takeaways

• Research infrastructure is eight stages with contracts; the invariants (causality, purge discipline, parity, immutability) matter more than the tools.
• Vectorized simulation is for screening and carries a standing look-ahead debt; event-driven is for verdicts. Know which regime you’re in every time you write .shift(1) — or fail to.
• Reproducibility is four cheap habits: configs in files, seeds everywhere, datasets fingerprinted, outcomes registered. None costs more than minutes; all pay for months.
• Notebooks explore, scripts ship; the promotion ritual keeps the two from drifting apart.
• Your production stack already embodies most of this anatomy — the honest audit shows monitoring and research-side statistics as the open fronts, which is exactly where Modules 19, 24, and 17 aim.

References

• Marcos López de Prado — Advances in Financial Machine Learning, ch. 1–2 (research factories, backtest hygiene), ch. 7 (ensemble/registry thinking)
• Stefan Jansen — Machine Learning for Algorithmic Trading, 2nd ed., ch. 1–2 (pipeline anatomy, data hygiene)
• Robert Carver — pysystemtrade (open-source full-stack reference; read system/ layout against this module’s anatomy)
• paperswithbacktest — awesome-systematic-trading (engine survey context)
• vectorbt — documentation (the vectorized philosophy, taken to its extreme)
• Next in sequence: Part II begins — Module 4 — Market Data Engineering