Module 24 — MT5 Platform Engineering & Live Deployment

Part V · Production · Priority 🎯 Core · Status: Draft v0.1 Prerequisites: Module 1, Module 19, Module 22, Module 23

Overview

Modules 17–21 produced models whose statistics survive honest validation. This module is the bridge between a validated model and money at risk: the platform engineering that guarantees the system you tested is the system that trades. Two deployment architectures exist on MT5 — an ONNX model embedded inside the Expert Advisor, or an external Python service talking to a thin EA over a socket — and choosing knowingly matters more than mastering either one’s details. Most live ML failures are not modeling failures: they are plumbing failures — a feature computed differently on the live path, an orphaned position after a crash-restart, a slow memory leak discovered in week three.

The accounting frame is the house style: a live deployment is a set of internal controls. Your trade journal is the general ledger, the broker’s server holds the counterparty’s books, and reconciliation is the month-end close that proves they agree. Everything in this module is either building that ledger, closing against it, or keeping the lights on while it runs.

House position: this is your capstone module. Your production supervisor already implements hardened Architecture B — ahead of most retail and much institutional practice — so the depth below goes to what remains: bit-level feature-parity proof, drift telemetry, explicit latency budgets, and the ONNX-in-EA fast path as an optional lane.

Two Deployment Architectures

💡 Idea: picking a deployment lane is a make-or-buy decision. Architecture A keeps the whole production chain inside one audited artifact (the EA — your own books). Architecture B hires a specialist department (Python) and accepts interdepartmental mail delays and the risk the department is down when the order arrives. Neither is wrong; being unaware of which one you run is.

Architecture A — Embedded ONNX inside the EA

The full loop lives in one process: train in Python, export the fitted model to ONNX, and let MQL5 host the runtime.

• Train in Python (scikit-learn, XGBoost, PyTorch — whatever Part IV produced).
• Export with skl2onnx, tf2onnx, or torch.onnx.export, using dynamic batch axes and float32 tensors throughout.
• Load once in OnInit via OnnxCreate("model.onnx", ONNX_DEFAULT), declare tensor shapes with OnnxSetInputShape.
• Infer per closed bar with OnnxRun inside OnTick/OnTimer.
• Release the model in OnDeinit with OnnxRelease.

Pros: minimal latency (inference is a function call, no inter-process hop), and — uniquely — the model runs inside the Strategy Tester, enabling true model-in-the-loop backtests. Cons: MQL5’s ONNX runtime supports a subset of operators, so exports must be verified visually in Netron before any MQL5 is written; and tensor discipline is float32 end to end.

# Export a trained Meta-model to ONNX so it can live inside an MT5 EA.
import torch

model.eval()                                   # freeze dropout/batchnorm state
example = torch.randn(1, N_FEATURES, dtype=torch.float32)   # ONNX wants f32

torch.onnx.export(
    model,
    (example,),
    "meta_r0.onnx",
    input_names=["features"],
    output_names=["probabilities"],
    dynamic_axes={                    # let the EA pick batch size at runtime
        "features": {0: "batch"},
        "probabilities": {0: "batch"},
    },
    opset_version=17,                 # pin the opset the MT5 runtime accepts
)
# Before writing ANY MQL5: open meta_r0.onnx in Netron. Confirm every operator
# survived the export and inputs display as float32 [batch, N_FEATURES].

The sklearn/XGBoost counterpart is skl2onnx.convert_sklearn(clf, initial_types=[("features", FloatTensorType([None, N_FEATURES]))]) — same rules apply: float32 inputs, pinned opset, Netron inspection.

Minimal MQL5 lifecycle — load once, infer per closed bar, release cleanly:

Code shown as plain text (no highlighting available for “mql5”).

// ONNX lifecycle: load once in OnInit, infer per closed bar, release cleanly.
long  model_handle;
float features[1][16];     // reused buffer - never reallocated per tick
float scores[1][2];

int OnInit() {
   model_handle = OnnxCreate("meta_r0.onnx", ONNX_DEFAULT);
   if(model_handle == INVALID_HANDLE) return INIT_FAILED;
   long shape_in[] = {1, 16};            // use -1 first dim if batch dynamic
   if(!OnnxSetInputShape(model_handle, 0, shape_in)) return INIT_FAILED;
   return INIT_SUCCEEDED;
}

void OnDeinit(const int reason) {
   if(model_handle != INVALID_HANDLE) OnnxRelease(model_handle); // mandatory
}

void OnTimer() {
   BuildFeatures(features);              // identical math to training path
   OnnxRun(model_handle, ONNX_NO_CONVERSION_CHECK, features, scores);
   ActOnScore(scores[0][1]);             // thresholds owned by Module 19
}

Architecture B — External Python service bridge

A thin MQL5 wrapper handles orders and housekeeping; everything intellectual lives in a Python process reachable over ZeroMQ, a raw socket, or a named pipe. The EA ships raw prices or prebuilt features; Python returns signals. Pros: unlimited model complexity (sequence models, ensembles, anything), hot-swappable models without recompiling the EA, and rich logging in the language your research stack already speaks. Cons: serialization and network latency (fine at seconds-and-slower cadence, wrong for tick scalping), and every hop is an independent failure point that needs supervision — process crashes, socket hangs, partial writes.

Choosing between them

Hybrid pattern

The lanes combine: ONNX fast-path inference inside the EA decides trades, while a Python sidecar watches telemetry, raises alerts, and fires retraining triggers. The EA never depends on the sidecar being up — monitoring degrades gracefully instead of taking positions down with it. This is the natural end-state for a pipeline that starts at B and wants sub-M5 ambitions later.

The Deployment Gate

Figure: a validated model does not go live directly — it passes through a sequence of gates, each able to reject it back to research. The lane choice happens first, but every downstream gate is identical regardless of lane.

flowchart TD
    artifact[(Part IV model<br/>bundle, validated)]
    choice{Deployment lane?}
    laneA["Lane A: ONNX<br/>inside EA"]
    laneB["Lane B: Python<br/>service bridge"]
    laneC["Lane C: hybrid<br/>ONNX + sidecar"]
    parity{Golden-dataset<br/>parity suite green?}
    tester{Tester replay<br/>matches backtest?}
    paper{Paper window meets<br/>backtest expectations?}
    golive[Go-live checklist<br/>signed]
    live[Live trading]
    monitor[Drift monitors<br/>schema-v6 snapshots]
    retrain[Retrain trigger to<br/>Module 19 loop]
    fix[Back to research:<br/>hunt the divergence]

    artifact --> choice
    choice -->|"sub-second cadence"| laneA
    choice -->|"seconds or slower"| laneB
    choice -->|"fast path + oversight"| laneC
    laneA --> parity
    laneB --> parity
    laneC --> parity
    parity -->|"fail"| fix
    parity -->|"pass"| tester
    tester -->|"disagreement"| fix
    tester -->|"agreement"| paper
    paper -->|"alarm fired"| fix
    paper -->|"clean 4 weeks"| golive
    golive --> live
    live --> monitor
    monitor -->|"drift confirmed"| retrain
    retrain --> artifact

    classDef data fill:#e8f0fe,stroke:#4a86e8
    classDef process fill:#f3f3f3,stroke:#888
    classDef decision fill:#fff4d6,stroke:#d6a300
    classDef risk fill:#fde8e8,stroke:#c0392b
    classDef ok fill:#e6f4ea,stroke:#2e7d32

    class artifact data
    class laneA,laneB,laneC,golive,monitor,retrain process
    class choice,parity,tester,paper decision
    class fix risk
    class live ok

How to read this:

• The three lanes converge immediately: architecture choice changes where work happens, never whether the gates apply.
• Every amber diamond can reject downward to research — the cheap death. Only signed human judgment (go-live checklist) opens the green state.
• The bottom loop is Module 19’s drift machinery: live telemetry feeds retrain triggers, and the cycle restarts from a validated artifact, never from a hot-patched one.

The Feature-Parity Problem

This gets its own section because it is the #1 documented cause of live ML failure: the model is fine, the export is fine, but the live path computes slightly different features than the training path did — and the model, exquisitely sensitive to its inputs, quietly produces different probabilities than your backtest assumed. Nothing crashes. The equity curve just stops resembling the validation report, and you find out in P&L.

Four things must match exactly between the Python training pipeline and the live inference path:

1. Indicator math. Same formula, same warm-up length, same NaN policy. A 14-period RSI using Wilder smoothing in training and cutler-style in MQL5 is a different feature wearing the same name.
2. Candle indexing conventions. Is index 0 the oldest bar or the newest? Does the live path ever see the forming bar, while training saw only closed bars? An off-by-one here shifts every feature one bar into the future — instant look-ahead on the live side, invisible in logs.
3. Rounding and formatting rules. Pip-rounding before or after differencing, decimal places carried through log transforms, float printing truncation when values cross a serialization boundary. Tiny biases compound across 50 features.
4. Broker-GMT-offset handling. Session filters, day-of-week features, and rollover boundaries all depend on the clock. If training read timestamps as UTC and the live terminal reports broker-local time shifted by +2/+3 hours, every session-conditioned feature is computed for the wrong part of the day.

📌 Convention: the broker-GMT-offset rule is recorded once — in the Module 1 context sheet and in one repo config value — and asserted everywhere else. Every parity test, session filter, and label builder reads that single value. Nothing anywhere hard-codes “+2” or “+3”; a DST change is a one-line config edit plus a rerun of the parity suite, not a bug hunt.

The defense is a golden-dataset parity harness: freeze a window of historical bars, push the same bars through both paths, and require bit-comparable features and probabilities. Stage 1 compares features; stage 2 compares model outputs given identical features. Both run as CI gates — a failing parity test blocks deployment the same way a failing unit test blocks a merge.

# Golden-dataset parity gate: same bars through both paths must agree.
import numpy as np
import onnxruntime as ort

bars          = np.load("golden/bars_2023Q4.npy")        # frozen OHLCV fixture
py_features   = build_training_features(bars)            # your polars pipeline
py_probs      = bundle.predict_proba(py_features)        # training-side model

live_features = np.load("golden/live_path_features.npy") # dumped by EA/service
ort_sess  = ort.InferenceSession("meta_r0.onnx",
                                 providers=["CPUExecutionProvider"])
onnx_probs = ort_sess.run(
    None, {"features": py_features.astype(np.float32)}
)[0].astype(np.float64)

feat_gap = np.abs(py_features - live_features).max()     # stage 1: features
prob_gap = np.abs(py_probs - onnx_probs).max()           # stage 2: outputs
print(f"feature gap {feat_gap:.3e} | probability gap {prob_gap:.3e}")

assert feat_gap < 1e-10, "PARITY FAIL: indicator math differs - stop here"
assert prob_gap < 1e-06, "PARITY FAIL: export/runtime changed the model"
# -> feature gap 0.0e+00 | probability gap 3.2e-07 : both gates green

⚠️ Pitfall: float64 Python vs float32 ONNX is a one-sided mismatch. Converting float64 features to float32 loses roughly seven decimal digits — harmless for well-separated probabilities, fatal when a score sits near its decision threshold, where a 1e-7 nudge flips a trade. Set parity tolerances accordingly (features tighter than outputs) and keep a threshold margin: from Module 19, never deploy a model whose profitable region hugs the 0.50 boundary within float32 noise.

Resource Lifecycle Discipline

Long-running EAs die of accumulation, not of any single error. Three habits keep a model-EA alive for months:

• OnnxRelease in OnDeinit. Every OnnxCreate allocates runtime memory outside the garbage collector’s reach. Skipping the release leaks a model-sized block on every recompile, timeframe switch, and parameter change — each of which calls OnDeinit.
• Buffer reuse inside OnTick/OnTimer. Declare feature and score arrays once at global scope and overwrite them per cycle. Allocating fresh arrays on every tick fragments memory until the terminal throttles or dies — the classic “worked fine for two weeks” failure.
• Memory-guard tripwires. Log the process working set hourly; alert when it grows beyond a ceiling (say, 1.5× steady-state); shut down gracefully — flatten nothing, just stop opening — when a hard limit trips. Your supervisor already implements the hard-trip version (exit code 3); the EA-side habit is the same idea applied to terminal memory.

Latency Budgets — A Worked Example

Every deployment lane has a deadline equal to its decision cadence, and each pipeline stage spends part of it. Budgets are set per stage, measured, and treated like a cost line — overspend is visible before it hurts. For an M5 system, one bar closes every 300,000 ms, so the entire decide-and-act cycle shares that budget:

Hand-check: 20 + 2 + 30 + 5 + 80 = 137. Meaning: at M5 cadence, even the bridge architecture is effectively free — you have 2,189× headroom. The same stack against a tick-scalper’s 250 ms budget tells the opposite story: 137 ms burns 55% of the budget before counting tick processing itself, and the network hop’s tail (a 30 ms median routinely spikes past 200 ms) breaches the deadline outright. Strip the bridge (Architecture A removes serialize + network): 105 ms of bounded cost — 42% of budget with no variance term — which passes. This is why lane choice follows cadence: seconds-and-slower → B is fine; sub-second → A or hybrid.

Strategy Tester for Model-EAs

Only Architecture A puts the model inside the tester, which buys a distinct capability: model-in-the-loop backtesting — the same binary that will trade live replays history under broker-grade fill simulation. Use every-tick real-tick mode; OHLC-projection modes fabricate intrabar paths that defeat the point. Reconcile tester equity against your Python engine (the Module 5 ritual) — disagreement is a finding, usually a feature-parity leak caught for free.

Two cautions keep the tester honest:

• Genetic optimizers will find lucky thresholds. If you expose the model’s decision threshold to the optimizer, the genetic search will discover the one cutoff that memorizes the sample’s noise. Thresholds belong to Module 19’s validation discipline — chosen on OOS evidence, frozen before the tester ever runs. Expose only execution parameters (session windows, slippage ceilings) to optimization, and demand plateaus over peaks (Module 5).
• Tester fills ≠ live fills. Even real-tick mode cannot know your order’s market impact or requote behavior at news. Treat tester results as the execution-reality anchor, not the verdict.

Operations Runbook

The runbook is written before go-live, rehearsed in paper trading, and updated after every incident. Its sections:

VPS selection and latency. Host within the same metro/datacenter region as your broker’s trade servers; measure ping continuously and alarm on regression (a jump from 2 ms to 40 ms means routing changed). Cheap VPS oversold on CPU causes inference-time variance — benchmark under load, not idle.

Terminal auto-restart behavior. Know what your EA does in OnInit after a crash: it must assume nothing about open positions until reconciled against the broker (below). Disable popup spam; enable crash logging. Test restarts deliberately — an untested recovery path is a hypothesis, not a control.

Connection-loss semantics. MT5 queues nothing for you: during a disconnect, stops still execute server-side (if set as server-side stops), pending logic does not run, and reconnects deliver a state you must diff against your journal. Decide in advance: does the strategy tolerate a blind gap, or must the kill-switch fire?

Magic-number and account segregation. One magic number per strategy per symbol, hardcoded from config — reconciliation filters on it, so collisions corrupt the close. Never share a demo/live account between experimental and production EAs; blast-radius control is account-level before it is code-level.

Order-state reconciliation and idempotency. After any restart: read broker positions, diff against the journal, and resolve each delta by a fixed rule — journal is authoritative for intent, broker is authoritative for existence. Unknown broker tickets get adopted or flagged, never silently ignored; journal tickets missing at the broker get marked closed with reason. Every order carries an idempotency key (magic + signal timestamp) so a retry after timeout cannot double-fire.

Kill-switch and manual override. A dedicated kill EA (or cockpit button) flattens all positions for a magic number and blocks new entries; every operator (including future-you, sleepy) knows the one-action emergency stop. Manual interventions get journaled like algorithmic ones, or the audit trail lies.

Heartbeat and alerting. Silence is the dangerous state, not errors — a dead supervisor looks identical to a quiet market. Push a heartbeat on a timer (Telegram is the retail standard) and alarm on absence. The sketch below is the supervisor-side loop; the EA mirrors it:

# Supervisor loop sketch: heartbeat out, reconcile journal vs broker truth.
import time

RECONCILE_EVERY_SEC = 30

while supervisor_running:
    now = time.time()

    if now - last_heartbeat > HEARTBEAT_INTERVAL_SEC:
        telegram_send(f"[{ACCOUNT}] alive | equity={equity():.2f} "
                      f"| open_tickets={len(journal.open_tickets())}")
        last_heartbeat = now

    if now - last_reconcile > RECONCILE_EVERY_SEC:
        broker_open = mt5.positions_get(symbol=SYMBOL) or []
        broker_ids  = {p.ticket for p in broker_open}
        journal_ids = {t.ticket for t in journal.open_tickets()}
        for vanished in journal_ids - broker_ids:
            journal.mark_closed(vanished, reason="missing-at-broker")
        for unknown in broker_ids - journal_ids:
            adopt_or_flag(unknown)     # orphan adoption, never silent ignore
        last_reconcile = now

    time.sleep(1)

Telemetry back to Python — closing the loop. Ship live inference vectors (your schema-v6 snapshot format) and fills back to the research environment on a schedule. Drift monitors (Module 19) join these snapshots against training references daily; a PSI breach becomes a retrain trigger, which re-enters the deployment gate at the top of this module. Without this leg you are flying blind between paper-run and post-mortem.

Deployment checklist. Executed in order; each step gates the next:

1. Golden-dataset parity suite green in CI (feature and probability gates).
2. Strategy Tester replay reconciled against the Python backtest, deltas explained in writing.
3. Latency budget measured per stage on production hardware, within limits.
4. Four-week demo paper run meeting backtest expectations, divergence alarms armed and quiet.
5. Chaos drills executed with documented recoveries (below).
6. Go-live review: checklist signed by the accountable human — dated, with the model hash.

🔧 For your pipeline: your src/live supervisor (SQLite WAL journal authority, gap re-proofing, orphan adoption, memory guard exit-3, Streamlit cockpit) is already a hardened Architecture B — ahead of most retail and much institutional practice. This module therefore concentrates on what you don’t have yet: a bit-level golden-dataset parity harness (Section above — build it first), drift dashboards wired to your schema-v6 snapshots (Module 19), explicit latency budgets per cycle stage, and Architecture A kept as an optional future lane if you ever push below M5. Capstone: walk one Part-IV model through the parity suite → 4-week paper run → signed go-live checklist.

Pitfall Checklist

The six documented ways model deployments fail on MT5 — each with its signature symptom and standing defense:

Testing It Honestly

Parity fixtures are CI gates, not documentation. The golden dataset regenerates only by deliberate version bump; both paths consume the identical fixture; a green suite is a merge requirement. When indicator code changes, the fixture proves the change was intentional on both sides simultaneously.

Chaos drills rehearse failure on your schedule, so live chooses no schedule of its own. Each drill has a documented expected recovery; if reality diverges from the doc, the recovery code is wrong, not the drill.

Run each drill quarterly and after any recovery-code change. An unrehearsed kill-switch is a rumor.

Hands-On Project

Deliverable: one Part IV model deployed through Architecture A or B on a demo account, with a full monitoring stack (src/deploy/) and a written operations runbook (docs/runbooks/live-<model>.md).

Tasks:

1. Choose the lane deliberately: write the one-page decision memo (cadence, model complexity, tester needs) before any code.
2. Export the model (A) or wrap the service contract (B); if A, verify the ONNX graph in Netron and record supported operators.
3. Build the golden-dataset parity harness and wire it into CI; regenerate fixtures once, deliberately.
4. Fill the latency-budget table with measured per-stage numbers on production hardware.
5. Paper-trade the deployment on demo for four weeks with divergence alarms (live vs backtest expectations) and drift dashboards fed by schema-v6 snapshots.
6. Execute all three chaos drills; document actual vs expected recovery for each.
7. Write the operations runbook (all runbook sections above) and obtain the go-live sign-off.

Acceptance criteria:

• Parity suite green in CI: feature gap < 1e-10, probability gap < 1e-6.
• Lane-choice memo exists and cites cadence, complexity, and tester requirements.
• Measured latency budget within limits; tick-cadence systems justify lane A explicitly.
• Four-week paper run completed with zero unexplained divergences.
• All chaos drills executed with documented, matching recoveries.
• Runbook complete (kill-switch, reconciliation, heartbeat, escalation) and checklist signed with model hash.

Key Takeaways

• The bridge from validated model to money-at-risk is platform engineering, and the #1 killer is not the model — it is the live path computing different features than training did.
• Two architectures exist; choosing knowingly by cadence and model complexity matters more than mastering either: bridge for seconds-plus and heavy models, embedded ONNX for speed and Strategy Tester compatibility.
• Bit-level golden-dataset parity — same bars, both paths, comparable outputs, enforced in CI — converts the silent killer into a loud test failure.
• The timezone rule lives in exactly one place and is asserted everywhere; session features computed at the wrong hour are indistinguishable from regime change until they cost money.
• Resources are manual in MQL5: release every model handle, reuse every buffer, tripwire memory before the terminal does it for you mid-position.
• Optimizers tune execution, never thresholds — thresholds are Module 19 property, frozen before the Strategy Tester opens.
• Reconciliation is a monthly-close discipline running every 30 seconds: journal authoritative for intent, broker authoritative for existence, deltas resolved by written rule.
• A deployment goes live only through the gate: parity green → tester agreement → four clean paper weeks → chaos drills documented → signed go-live checklist.

References

• MQL5 Documentation — Preparing ONNX Models / ONNX functions (OnnxCreate, OnnxSetInputShape, OnnxRun, OnnxRelease)
• MetaQuotes — MetaEditor Help: Machine Learning (ONNX integration workflow)
• MetaQuotes — MetaTrader5 Python package (bridge-side data and trade API)
• Lutz Roeder — Netron (ONNX graph inspector; pre-MQL5 export verification)
• samikhan48 — ML-Signal-Trader-MT5 (community reference: ONNX CNN/SVM inside an EA)
• Anaswar-ash — MT5-PY-AI-Tbot (community reference: full Python + MQL5 + ONNX framework)
• Next in sequence: Module 25 — Portfolio Construction & Strategy Combination