01-projects/investing

strategy pipeline architecture v0

2026-05-29·project·status: spec-pending-founder
systematic-tradingbacktest-enginestrategy-discoverypipeline-architecturehedge-fund-disciplineholdoutensembles

Strategy-Discovery & Execution Pipeline — Architecture (v0)

Greenfield architecture for a strategy-agnostic pipeline that DISCOVERS trading strategies, VALIDATES them honestly, and EXECUTES the survivors on Alpaca paper with hedge-fund position discipline. Synthesized from a vault survey (existing execution rails, validation doctrine, anchor feeds), three research vectors (backtest-infra build-vs-buy, deflated-Sharpe / multiple-testing, position sizing & in-position management), and an adversarial scoring of three full architecture proposals. The winning spine is "buy the vectorized core, build the RDCO discipline shell" (lean-vectorbt), grafted with the two highest-leverage salvages the red-team flagged: the import-level sealed-holdout boundary with auto-counted N and the structural economic-prior + per-family null gate that culls story-less candidates before they consume compute — plus the red-team's load-bearing scope correction: route the discovery swarm at large-sample liquid-universe data, NOT the rare-state chip cycle.

The reframe — strategy-discovery pipeline, Markov is one pluggable candidate (not the project)

The founder's ask is a system for finding and running winning strategies, not a Markov project. Markov earns its place only by winning on the same harness as momentum, mean-reversion, vol-regime, and breakout — or it doesn't promote. The vault already proved this discipline internally: the 2026-05-29 Markov requirements doc honestly concluded the chip capital cycle is too slow to be a Markov state machine (~4-6 cycles in 40yr cannot fit a transition matrix; v1.1 backtests showed buy-and-hold beat every mechanical de-risk variant by 19-91pp because the cycle's stress states ARE the V-bottoms the thesis says to BUY). The Markov work survives as exactly one candidate: a vol-regime exposure overlay on SPY (a Layer-1.5 throttle), ring-fenced away from the conviction core. That is the template for the whole pipeline: strategies are plugins; the engine and the gates are the product.

Three structural truths shape everything below, and they are honest tradeoffs, not features:

  1. The binding scarcity is rare-regime sample size, not data granularity. Daily bars are all this horizon needs (position/swing, not intraday). The constraint is that CRISIS/STRESSED episodes are few (~5-6 in SPY history; ~4-6 chip cycles in 40yr). This is the single fact that disciplines the discovery scope: run the swarm where the sample is large (liquid index/ETF surfaces, cross-sectional equity momentum over hundreds of names) and keep rare-state cyclicals as hand-built conviction theses, off the swarm entirely. The discovery loop is mechanically forbidden from being pointed at a 5-sample universe and asked to "discover" a winner — that is data-mining by construction.

  2. The most expensive lesson in the vault is "no mechanical exits on the conviction core." Any discovery harness that re-finds "sell on -15% drawdown" as alpha on the conviction names is re-finding a disproven rule. The strategy interface makes this mechanical: a conviction-core strategy literally cannot register a price-based exit. Dual buy-and-hold benchmarks are mandatory so the pipeline can never "discover" something that loses to doing nothing.

  3. Ensembling is complementary, not a spurious-correlation cure. Averaging horizons per symbol reduces horizon-fragility and variance — real benefit — but you can ensemble overfit signals into a confidently overfit ensemble, and the ensemble is itself selected from the search. The three actual defenses are wired as GATES not reports: (a) a sealed OOS holdout the search never touches, (b) a multiple-testing penalty (Deflated Sharpe + PBO), and (c) an economic prior. Ensembling layers on top of already-promoted survivors.

Architecture — the component spine

Eight modules, left to right, in a new repo rdco-quant/ (PR-only, paper-only, with a running implementation-notes file per feedback_implementation_notes_sub_agent_pattern). Each box names how it connects.

                    ┌─────────────────────────────────────────────────────────────┐
                    │  HOLDOUT (sealed): import-level wall — discovery CANNOT read  │
                    └─────────────────────────────────────────────────────────────┘
                                              ▲ touched once, at the end, per candidate
                                              │
 [1 DATA FEED] ──▶ [2 BACKTEST ENGINE] ──▶ [4 DISCOVERY/SEARCH] ──▶ [5 VALIDATION GATE] ──▶ [6 SIZING] ──▶ [7 EXECUTION] ──▶ [8 MONITORING]
   Stooq+yf+VIX        vectorbt + thin          sweep over             economic-prior         vol-target      Alpaca paper        attribution
   (backtest hist)     custom shell:            family×symbol×          + DSR + PBO            spine +         fractional          ledger +
   Alpaca SIP          cost model,              horizon×params;        + dual-BH;             earn-your-      DAY LIMIT           rollup +
   (live signals)      no-lookahead             auto-emits N           survivors only         Kelly +         (never OPG);        live-calibration
        │              mutation test                  ▲                pass through           heat/corr       sleeve-routed       drift watch
        │                   ▲                         │                                       caps            mgmt                     │
        └───────────────────┴──── [3 STRATEGY INTERFACE] (the pluggable Strategy ABC) ───────────────────────────────────────────────┘
                                   momentum | mean-rev | vol-regime(Markov) | breakout | (conviction theses, hand-built, OFF the swarm)
  1. DATA FEED (data/) — Two-source split. BACKTEST history: Stooq daily CSV (deep, delisted archive) primary + yfinance (adjusted-close cross-check, research-cache-only, never in the live loop) + CBOE VIX. Cached to the existing layout ~/.claude/state/backtest-cache/<run-id>/bars/<ticker>.parquet. LIVE signals: Alpaca free 15-min-delayed SIP (daily-close horizon makes the delay irrelevant) via ~/.claude/scripts/alpaca-env-paper.sh + alpaca-venv/. Survivorship-free universe membership via Wikipedia page-revision snapshots + iShares holdings, with a ticker-alias map (FB→META). A validate_bars() pass asserts continuous coverage / no silent gaps. Feeds → engine (history) and execution (live bars).
  2. BACKTEST ENGINE (engine/) — vectorbt (OSS, Numba-vectorized) wrapped by a thin run_backtest(strategy, bars, cost_model) adapter. The adapter, NOT vectorbt, owns the three things no OSS engine does for RDCO: (a) walk-forward orchestration + leave-one-out aggregator, (b) anchor/phase-driven exit logic for conviction strategies, (c) the shared cost model (maker-limit fill, slippage bps, short-term-cap-gains + wash-sale-aware NET-return layer) and the no-lookahead future-leak mutation test. Consumes ← strategy interface; emits → discovery + validation.
  3. STRATEGY INTERFACE (strategies/base.py) — the pluggable Strategy ABC every family implements (detailed below). Decouples the engine from the strategy so Markov, momentum, mean-reversion, vol-regime all run on identical rails.
  4. DISCOVERY / SEARCH (discovery/) — vectorbt vectorized sweep over family × symbol/universe × horizon × param-grid, running ONLY on the TRAIN/VALIDATION partition, auto-incrementing a cumulative trial counter N. Ranked output feeds → validation gate.
  5. VALIDATION / HOLDOUT GATE (validation/) — the sealed-holdout manager (import-level wall) + the three gates (economic-prior, DSR/PBO multiple-testing penalty, dual buy-and-hold). Mechanically forbidden from reading holdout during search. Only triple-pass survivors → sizing.
  6. SIZING (sizing/) — vol-targeting spine (weekly rebalance, no-trade band) + earn-your-Kelly graduation (fixed-fractional 1% → quarter-to-half Kelly only after OOS calibration) + portfolio heat cap + correlation-cluster cap, all in the locked R-unit frame (1R=$5k, 2R/name, 4R/bucket — see open question on the 3R drift). Feeds → execution.
  7. EXECUTION (execution/) — extends the existing alpaca-*-deploy.py skeleton (account guards, R-sizing, result JSON) but STANDARDIZES on fractional-notional DAY LIMIT orders (fixes the market-order caveat the survey flagged). Tags every fill into positions/_attribution-ledger.csv. Founder-gated by canonical DECISION-format / explicit deploy verb. Feeds → monitoring.
  8. MONITORING (monitor/) — daily cron computes signals/sizing-deltas, writes a state file, surfaces a channel message ONLY on actionable change (no daily nag). Builds the specced-not-built attribution-rollup.py for honest per-strategy P&L across the single undifferentiated Alpaca book; live-vs-backtest calibration drift watch feeds back to validation.

The pluggable strategy interface — the contract every strategy implements

The square-peg fix is one ABC in strategies/base.py that the engine, discovery swarm, sizing, and execution all program against, so the engine never knows whether it is running Markov or momentum.

REQUIRED METHODS:

REQUIRED DECLARATIONS (frozen at registration, enforce honest reporting):

Markov plugs in as ONE family: signal() emits a gross-exposure multiplier on the SPY+satellite sleeve, sleeve=overlay, exit_class=mechanical (legal because it never touches conviction-core tickers), prior_strength=high (vol clustering), null_test() returns the FR-3 LR + dwell-time result. It competes on the SAME rails as everything else — if it can't beat momentum through the same gate, it doesn't promote. No square peg.

Discovery loop + anti-overfitting

For each registered Strategy, the loop sweeps param_space() × universe × horizon-set (daily/weekly/monthly holding) using vectorbt's vectorized engine on the TRAIN/VALIDATION partition only (holdout physically withheld). The sweep is bounded by a pre-registered budget (e.g. ≤2000 cells/run) so N is known a priori. Output: a ranked table of {strategy, symbol, horizon, params, in-sample Sharpe, trial_id}. Ranking is NOT by raw Sharpe.

Three gates, applied in order, ALL must pass — these are GATES not reports:

Only candidates clearing all three AND beating BOTH mandatory buy-and-hold benchmarks (universe + broad index) advance to the sealed holdout.

How ensembles slot in (honest): after individual survivors exist, multi-horizon-per-symbol averaging is constructed as its OWN candidate and re-scored on the holdout with its own DSR (counting the number of ensembles tried). Ensembling is a variance-reducer layered on validated edges — explicitly NOT a way to rescue cells that failed the gates, and explicitly NOT a spurious-correlation defense.

Validation discipline — walk-forward + ring-fenced OOS holdout + the go/no-go bar (the "wet rehearsal")

THREE-PARTITION DISCIPLINE, sealed before any search touches a byte:

The holdout-exhaustion risk (the red-team's sharpest flaw, surfaced honestly): touch-once-then-retire means the holdout is a depleting resource, while the swarm wants to emit candidates indefinitely. This is unmeetable on a 5-sample rare-regime universe — which is precisely why the architecture routes discovery at LARGE-sample liquid surfaces where holdout segments are long enough to amortize many candidates, and keeps rare-state cyclicals as hand-built theses validated through the existing v2 harness (one thesis, one holdout touch), never through the swarm.

The go/no-go promotion bar (wet rehearsal, pre-registered before looking at OOS): mirrors the Markov three-bar discipline, generalized. Bar 1 = the per-family null_test() passes (structure vs iid). Bar 2 = calibration holds out-of-sample within CI (predicted base rates match realized). Bar 3 = risk-adjusted improvement AFTER costs and AFTER tax, beating both buy-and-hold benchmarks across ≥2 leave-one-out stress episodes. ALL three pass on walk-forward → ONE holdout evaluation → only then is paper deploy a founder-gated DECISION. Pre-registering the bar before seeing OOS is the antidote to fitting the bar to the answer.

New mandatory report section vs today's harness: the MULTIPLE-TESTING LEDGER (N trials run cumulative, DSR, PBO, trial-Sharpe distribution) — the single missing piece between "validate one thesis" and "discover winners without fooling ourselves." Pre-register the expectation that MOST candidates should FAIL the gates; a pipeline that "discovers" many winners is suspect, and a clean negative result (nothing beats buy-and-hold-with-kill-switch after cost+tax) is a SUCCESS of the discipline, not a failure of the build.

Execution & sizing discipline

ENTRY. On a cleared signal at/after the daily close (15-min-delayed SIP is fine — not day-trading), translate target weights into order deltas ONLY when the delta exceeds a no-trade band (~20% drift) to avoid churn + wash-sale tripwires.

POSITION SIZING (three tiers).

IN-POSITION MANAGEMENT (split by sleeve — NOT one global rulebook).

PORTFOLIO RISK. Explicit heat cap (sum of distance-to-stop × size ≤ ~6-10% equity), correlation-cluster cap computed not implied (MU+SMH+SNDK+INTC = ~one semi-memory bet, share one bucket cap), −10% portfolio drawdown kill-switch, plus the vol-targeting spine as the portfolio-level de-risking mechanism (the architecturally correct place for "stop-loss-like" behavior — at the portfolio throttle, not position-level stops on the core).

ORDERS. Alpaca paper, fractional DAY LIMIT with maker bias, NEVER OPG/market-on-open (paper-sandbox OPG expires unfilled). The order builder REJECTS any market-on-open path (FR-7 assertion). Reuses the existing deploy-script skeleton corrected to limit/DAY.

TAX-AWARE. Longer holds are partly a tax optimization, so the backtest scores AFTER-TAX and net-of-cost (short-term-cap-gains drag is visible in the discovery ranking, not just asserted). Systematic re-entry within 30d of a loss trips the wash-sale rule and disallows the harvest, so cadence stays weekly-to-monthly on the satellite and effectively never-mechanical on the core — the horizon constraint is tax-aligned, not just risk-aligned.

GATE. Any live capital is founder-gated by canonical DECISION-format APPROVE or explicit "deploy" verb (structural agreement is NOT enough, per feedback_paper_trade_deploy_authorization); paper-only for now.

Build vs buy — the backtest-engine + data-feed recommendation

Backtest engine: BUY-the-core / BUILD-the-shell. Adopt vectorbt (OSS, Numba-vectorized) for the discovery sweep — it already solved the vectorized param-sweep over many symbols × horizons × variants, which a discovery-first pipeline lives or dies on; rebuilding that in custom pandas is a slow-to-write, slow-to-run maintenance sink (the two existing strategy-specific drivers prove custom-pandas works but is harvest-for-parts, not a generic engine). Commons-Clause license is fine for RDCO solo/personal-use per project_personal_license_boundary; re-check only if it ever became a shared product. Then BUILD a thin custom shell for the three things no OSS engine does well for RDCO: (a) walk-forward orchestration + leave-one-out aggregator, (b) anchor/phase-driven exits ("hold through cycle, exit only on fundamental anchor-break" is no price rule any engine models), and (c) the multiple-testing ledger + sealed-holdout + DSR/PBO gate (the actual moat).

Alternatives: backtrader — DROP (legacy/archived, Python 3.10+ friction, slow at scale); recommend amending the /investing:backtest-thesis SKILL.md line "vectorbt or backtrader" to "vectorbt (search) + thin custom WFA/exit/DSR shell; zipline-reloaded fallback; backtrader dropped." zipline-reloaded — KEEP DOCUMENTED as the fallback if vectorbt's vectorized model fights the held-through-cycle anchor-exit logic (it has native point-in-time/dynamic-universe handling). vectorbt-PRO ($25-500, native WFA) — DEFER; buy only if the manual WFA wrapper becomes the bottleneck (cost isn't the blocker per feedback_api_cost_budget_controlled, premature dependency is). NautilusTrader — order-book/sub-second overkill for daily equities; only later as a pre-live execution-realism check.

A correctness safeguard worth keeping from the custom-spine proposal: run a handful of canonical strategies through BOTH the vectorbt path and a tiny independent pandas reference, asserting returns/Sharpe match within tolerance — a cheap shadow-oracle on the hand-rolled cost/WFA layer. It does NOT validate the hard parts (point-in-time membership, anchor exits, the no-lookahead guard) — those need the dedicated mutation/golden-fixture tests.

Data feed: BUILD the thin stitching layer over FREE sources. Stooq (deep daily + delisted) + yfinance (adjusted-close cross-check, research-cache-only, never live — it's unofficial scraping, rate-limited, silently inconsistent) + CBOE VIX for the backtest; Alpaca free 15-min-delayed SIP for live paper; Wikipedia page-revisions + iShares holdings for survivorship-free universe membership. Total recurring cost $0. Do NOT buy Polygon (intraday-oriented, rate-limited, unneeded at daily horizon). Upgrade lever: Tiingo (~$10-30/mo) ONLY if the free-data validation pass keeps failing. Honest gap to flag: none of the free sources give true point-in-time fundamentals (market cap, float) — only point-in-time membership (Wikipedia). A universe rule referencing fundamentals ("market cap >$5B at window start") forces either a membership proxy or a paid CRSP/Compustat dependency; the harness surfaces this confession per-window rather than silently biasing the universe.

Phased build plan

Each phase is independently shippable + verifiable. Built as todo-file + dynamic /loop with a running implementation-notes file (feedback_todo_file_loop_vs_notion_queue + feedback_implementation_notes_sub_agent_pattern), PR-only, paper-only. Each phase maps to the V-model discipline: every component ships with its paired right-arm verification (the test written with the spec, not after).

PHASE 1 — THE SPINE (engine + interface + one strategy, end-to-end on the search partition). This is first because everything else consumes it. Wire the data feed (Stooq + yfinance + VIX cached to the existing layout, validate_bars no-gap test) → vectorbt adapter with cost model + future-leak mutation test → the Strategy ABC → reimplement ONE seed strategy (momentum, simplest) on the interface → produce a v2-format honest report (per-cycle table, leave-one-out, dual benchmarks, CI). No discovery, no execution yet. V-model pairing / verify: the report reproduces a known result; the future-leak mutation test fails-closed (leaking a future bar changes the result); the vectorbt-vs-pandas shadow oracle matches within tolerance.

PHASE 2 — THE GATE (the anti-data-mining moat, BEFORE any breadth). Sequenced before the swarm on purpose — a discovery loop without the gate is a slop cannon. Build the sealed HoldoutManager (import-level wall + touch-once-then-retire) + the cumulative auto-N trial ledger + Deflated Sharpe + PBO/CSCV + the economic-prior/null gate. V-model pairing / verify: feed a deliberately-overfit pure-noise strategy through a 100-cell sweep and assert the best cell is REJECTED by DSR (the canonical "did we build the overfitting defense correctly" test); feed momentum through and assert N is counted and the holdout is untouched during search (the access-boundary test raises on any holdout read).

PHASE 3 — THE SWARM (discovery breadth). Candidate-generator + fan-out backtest grid over seeds (momentum, mean-reversion, vol-regime/Markov, breakout) on TRAIN/VAL, capped trial budget, ranked survivor output through the Phase-2 gates + dual benchmarks. V-model pairing / verify: a full sweep produces a ranked survivor list with a multiple-testing ledger and at-most-the-budget N; Markov competes as one family with zero special-casing (the square-peg proof — it wins or loses on the shared rails); nothing reaches the holdout without clearing the gates.

PHASE 4 — SIZING + EXECUTION (paper, founder-gated). Vol-targeting + earn-your-Kelly + heat/correlation-cap modules → extend the deploy skeleton to fractional DAY LIMIT (drop market orders) → attribution-ledger tagging + attribution-rollup.py. V-model pairing / verify: the order builder REJECTS OPG/market; the ring-fence test (changing the overlay input leaves conviction-core dollars invariant); sizing never exceeds quarter-Kelly of the calibrated edge; a promoted survivor runs ≥2 weeks live-paper with live-emitted signals matching the backtest on the same days; the rollup shows per-strategy P&L. Live capital remains a separate founder-gated DECISION.

PHASE 5 — MONITORING + ENSEMBLES + LIVE-CALIBRATION WATCH (refinement). Daily cron (signals/sizing-delta, state file, actionable-only channel message) + multi-horizon ensembling on promoted survivors (re-scored on the untouched holdout with its own DSR) + a production calibration drift monitor (matrices/edges may be non-stationary). V-model pairing / verify: the cron emits state cleanly; an ensemble beats its constituents on the holdout; the live-calibration watch flags drift.

Open questions for founder

  1. Discovery scope — confirm the rare-state carve-out. The architecture deliberately runs the swarm only on large-sample liquid surfaces (cross-sectional momentum over hundreds of names, the SPY vol-regime surface) and keeps rare-state cyclicals (the chip cycle) as hand-built conviction theses validated through the existing v2 harness, NEVER through the swarm. This directly honors your "don't fit a square peg" instruction and the v0 doc's "chip cycle too slow to be the state machine" finding. Confirm — or do you want the chip-cycle phase tracker to also compete in the swarm (against my recommendation)?
  2. The R-frame drift to lock. The survey and Markov docs cite "2R/name"; memory-cycle v1.1 raised it to "3R/name." Before this is coded as a hard cap, which is canonical — 2R or 3R per name? (4R/bucket, 1R=$5k, −10% kill-switch are consistent across docs.)
  3. History backfill scope. Invest in the pre-1993 S&P 500 daily splice to roughly double the CRISIS/STRESSED sample (the single highest-leverage thing that moves the sample-size constraint the gates depend on), or accept the thinner 1993-present window and lean on the CRISIS shrinkage prior? The splice adds a data-stitching surface (index vs ETF, dividend treatment).
  4. Holdout exhaustion policy. Touch-once-then-retire makes the holdout a depleting resource. On a large-sample surface a long holdout amortizes many candidates, but at the project's lifetime scale we will eventually deplete it. Acceptable to plan a periodic holdout refresh (roll forward, sealing newly-aged data) as the only honest replenishment, or do you want a different policy?
  5. Tiingo trigger. Pre-authorize the ~$10-30/mo Tiingo upgrade to fire automatically if the free-data validation pass fails N times, or keep it as a per-instance founder ask?
  6. Dashboard-vs-sizing ambition for the Markov candidate. Per the v0 doc, Bar 3 (does-it-pay-after-costs) is the one most likely to FAIL for the vol overlay. If 1+2 pass but 3 fails, the honest outcome is a founder-facing regime DASHBOARD, not wired sizing. Is a dashboard-only outcome worth the build, or should the Markov candidate be deprioritized behind momentum/mean-reversion?

Related