Markov System Requirements (v0) — Vol-Regime Exposure Overlay

Left-arm top of the V. This is a PROTOTYPE-stage requirements artifact: honest, scrappable, rigorous. Every requirement below names its paired right-arm verification (the V-model discipline: the test is written with the spec, not after the code). The selected use-case is vol-regime-overlay — a 4-state realized-volatility-regime Markov chain on SPY used as a gross-exposure throttle on the existing memory-cycle book, NOT as a thesis-exit signal. Building/running anything here is a separate founder go; this defines what "good enough to turn on" means before a single line of engine code is written.

Why this use-case won — selection rationale + the chip-cycle "too slow" finding

Three Markov use-cases were generated and adversarially scored. vol-regime-overlay won (22/40) because it is the only finalist whose genuine edge points away from the one book the v1.1 thesis already proved it would lose on. The chip capital cycle — RDCO's lead conviction thesis — is honestly too slow to be the state machine of a Markov-timed system: there are only ~4-6 semi cycles in 40 years (too few to fit a transition matrix at all), the phase is observable only quarterly with months of confirmation lag, and the v1.1 backtests showed buy-and-hold beat every mechanical de-risk variant by 19-91pp precisely because the cycle's stress states ARE the V-bottoms the thesis says to BUY. So the chip cycle stays where it belongs: Layer 1, the strategic BET, untouched by mechanical timing. The vol regime is a faster Layer 1.5 throttle — daily transition step (above the excluded intraday floor), weekly-to-monthly realized rebalances (below the too-slow chip cycle) — that modulates EXPOSURE on a vol-targeted satellite + SPY-beta sleeve while the conviction core rides through. The candidate's own killer objection ("predictable-but-untradeable after costs") is a finding, not a dead end: if the edge does not survive costs, the same machinery becomes a founder-facing regime dashboard. That honest survivability is why it was selected over the two candidates that overlay the conviction book directly and would merely re-confirm a known null.

Concept of Operations

What the system does. Each trading day at the close, it deterministically labels SPY's current realized-volatility regime (CALM / NORMAL / STRESSED / CRISIS), maintains a transition matrix estimated walk-forward over history, and emits a single scalar: a gross-exposure multiplier for RDCO's equity book. It does not pick stocks, does not fire thesis exits, and does not trade intraday.

Who/what it serves. RDCO's own paper capital — specifically the memory-cycle v1.1 book (MU, SMH, SNDK, INTC) plus an SPY beta sleeve. The overlay touches ONLY the SPY sleeve and the non-conviction satellite portion. The conviction core is ring-fenced and never trimmed by the overlay (this is the load-bearing reconciliation with house doctrine, which killed mechanical exits).

The operating loop (one daily cycle):

1. Observe state. Pull SPY adjusted close; compute 21-day trailing realized vol RV_t; map to a regime via fixed frozen thresholds with 2-day hysteresis. Compute VIX close for the implied-vol cross-check.
2. Estimate transition probabilities. Using only data up to T (zero lookahead), maintain the regime transition matrix with the ≥30-obs-per-state discipline; CRISIS rows use a shrinkage prior toward STRESSED.
3. Size. Map the current regime to a target gross-exposure multiplier (CALM 1.0x / NORMAL 1.0x / STRESSED 0.6x / CRISIS 0.35x), applied via quarter-Kelly-disciplined sizing on the satellite + SPY sleeve only. De-risk asymmetry: CRISIS de-risk may fire within 2 days; re-risk requires the calmer state to hold ~5 sessions.
4. Execute. Translate the multiplier delta into maker/limit DAY orders on Alpaca paper (fractional notional, never market-on-open — per the OPG-unreliable lesson).
5. Monitor. Log current state, days-in-state, distance-to-next-threshold, and RV-vs-VIX divergence to the state file; surface a channel message to the founder ONLY when the multiplier would actually change (no daily nag).

System Requirements

Functional (FR) and non-functional (NFR). Each carries a Verified by: line naming its right-arm test.

Functional

FR-1 — State definition (deterministic, zero-lookahead labeling). Every historical and live bar is labeled by: (a) daily log return on adjusted close; (b) 21-trading-day trailing annualized realized vol RV_t = sqrt(252) * stdev(trailing 21 closes); (c) fixed absolute thresholds CALM <12% / NORMAL 12-20% / STRESSED 20-35% / CRISIS >=35%; (d) 2-consecutive-close hysteresis so a 1-day boundary spike that reverts does NOT record a transition. Thresholds are a hyperparameter frozen once on a pre-2010 calibration block. Verified by: unit test on the labeler (golden-input fixtures: a hand-checked synthetic series with known RV crossings + hysteresis reverts must produce the exact expected state sequence); plus a no-lookahead assertion test (label at bar T must be byte-identical whether or not bars > T exist in the input).

FR-2 — Transition-matrix estimation with >=30 observations per state. Estimate the 4x4 matrix by counting state(t)->state(t+1) over the in-sample window. Enforce the >=30-obs-per-state rule from the existing pipeline spec. CALM/NORMAL pass easily; STRESSED is marginal; CRISIS will FAIL the raw rule, so CRISIS-origin rows use a shrinkage prior toward the STRESSED row rather than being trusted raw. Report per-state observation counts alongside every matrix. Verified by: estimator test asserting (a) each row sums to 1.0 within tolerance; (b) any state with <30 raw obs is flagged AND its row is shrinkage-adjusted (not raw); (c) obs-count report is emitted and non-degenerate.

FR-3 — Random-walk / state-persistence disproof (the null-hypothesis gate). Before any trading logic, the estimated matrix must demonstrate genuine state persistence vs an iid null: diagonal self-transition probabilities materially above unconditional base rates, with a Markov-vs-iid likelihood-ratio (chi-square on transition counts) rejecting iid. Graft from runner-up (trend-vol-regime): convert the binary check into a dwell-time measurement — a true regime must show expected dwell materially longer than a memoryless geometric draw at the same self-transition rate. This is the #1 failure mode (disguised random walk) made mechanical and gating. Verified by: statistical test in the wet rehearsal — LR test rejects iid at p<0.01 AND empirical dwell-time distribution is significantly right-shifted vs the geometric null. FAIL here halts the build before trading logic is written.

FR-4 — Monte-Carlo forward simulation. From the current state, simulate forward regime paths via the transition matrix, mapping each regime-sequence to a forward-return / drawdown distribution using each regime's empirical return distribution (in-sample only). Output: a distribution, not a point estimate, over the positioning horizon. Verified by: simulation test asserting the MC stationary distribution converges to the matrix's theoretical stationary vector (eigenvector check) within tolerance, and that forward-drawdown draws from CRISIS-conditioned starts are stochastically dominated (worse) than CALM-conditioned starts.

FR-5 — Calibration to base rate. When the model conditions on the current state and predicts a forward distribution, realized base rates must match out-of-sample. If STRESSED predicts elevated forward drawdown, realized forward drawdowns from STRESSED entries must actually be elevated vs CALM entries, with confidence intervals (no 2-decimal point estimates on thin samples). This is the load-bearing honesty check inherited from the existing pipeline. Verified by: calibration report in the wet rehearsal — predicted-vs-realized within CI across the walk-forward, plotted per state; CRISIS-conditional claims explicitly carry wide CIs and an "n episodes" annotation.

FR-6 — Quarter-Kelly position sizing. The exposure multiplier is implemented through quarter-Kelly-disciplined sizing on the calibrated edge, applied to the satellite + SPY sleeve only. No full-Kelly. The multiplier schedule (1.0/1.0/0.6/0.35) is the regime-conditional target; quarter-Kelly governs how aggressively the satellite expresses it. Verified by: sizing-logic test asserting computed position never exceeds quarter-Kelly of the calibrated edge, AND that the conviction-core dollar allocation is invariant to the multiplier (the ring-fence assertion: changing the regime input must leave core positions untouched).

FR-7 — Maker/limit execution (OPG-unreliable lesson). All paper executions are fractional DAY notional limit orders with a maker bias. NO market-on-open / OPG orders (they expire unfilled in the Alpaca paper sandbox). Slippage is modeled in the backtest cost layer. Verified by: execution-layer test asserting every emitted order is type=limit, tif=DAY, notional-fractional, and that the order builder rejects any OPG/market-on-open path. Cross-ref [[feedback_alpaca_paper_opg_unreliable]].

FR-8 — Asymmetric de-risk / re-risk + conviction-core ring-fence. CRISIS de-risk may fire within 2 days; re-risk up requires the calmer state to hold ~5 sessions. The overlay modulates ONLY the SPY beta sleeve and the non-conviction satellite; the MU/SMH/SNDK/INTC conviction core is never trimmed by the overlay. This is the explicit guard against recapitulating the v1.1 mechanical-exit failure. Verified by: state-machine test on the asymmetry timing (de-risk latency <= re-risk latency by construction) AND the ring-fence assertion from FR-6 re-run at the system level; system-validation re-check that overlay-on vs overlay-off leaves core positions identical through the 2020 + 2022 V-bottoms.

Non-functional

NFR-1 — Walk-forward only / zero lookahead. Thresholds + hysteresis frozen on a pre-2010 calibration block. Transition matrix re-estimated using only data up to T, anchored walk-forward, stepping forward in ~3-month increments, never peeking ahead. Verified by: the no-lookahead assertion (FR-1) plus a walk-forward harness audit — the [[investing-backtest-thesis]] harness enforces point-in-time estimation; a deliberate "future-leak" mutation test must cause a detectable result change (proves the guard is live).

NFR-2 — Survivorship-free universe. SPY/VIX are index-level (no survivorship surface). The single-name overlay inherits the memory-cycle thesis's own point-in-time membership; it constructs no new universe. Verified by: harness assertion that the overlay universe is a pass-through of the thesis universe at each point in time (no retrospectively-curated ticker set introduced).

NFR-3 — Interpretability. State is a deterministic function of trailing closes — no latent inference, no black box. A human (and the founder) can re-derive today's state by hand from the 21 trailing closes and the frozen thresholds. Verified by: reproducibility test — a hand-computed RV on a sampled date matches the engine's labeled state; the regime dashboard surfaces the raw RV value, threshold distances, and the arithmetic, not just the label.

Data & Observability Requirements

Data sources, frequency, history depth

The load-bearing data ask is history depth, not granularity. Daily is all we need (no tick, no options chain — the secondary direct-vol mode is diagnostic-only). The binding scarcity is CRISIS/STRESSED episodes: SPY 1993-present gives ~5-6 genuine stress episodes (1998, 2000-02, 2008, 2011, 2020, 2022); the index splice roughly doubles that. The >=30-obs rule passes for CALM/NORMAL, is marginal for STRESSED, and FAILS for CRISIS without the deep backfill — which is exactly why CRISIS uses a shrinkage prior (FR-2). Verified by: data-acquisition test asserting (a) continuous daily coverage with no silent gaps across the full window; (b) per-state observation counts reported so the CRISIS scarcity is visible, not hidden.

Live observability (how we know which state we're in, in real time + the lag)

This is the candidate's strongest practical feature and the reason it beats the chip-cycle: the state is observable in near-real-time with a short, bounded, quantifiable lag.

• State known at each close. RV_t is a deterministic function of the trailing 21 closes — computable seconds after the close. No 13F wait, no earnings call, no judgment.
• Lag is structural and measurable. The 21-day trailing window lags a true vol shock by ~half the window (~10 trading days of "memory" of the old regime); the 2-day hysteresis adds up to 2 days of confirmation. So a regime CHANGE is confirmed with a ~2-12 day lag depending on abruptness. A sudden CRISIS (a -7% day dominates the 21-day stdev) is detected fast; a slow vol grind-up is detected slower.
• Honest lag caveat. Because RV is trailing, by the time we are confidently IN crisis a chunk of the drawdown has already happened. The overlay harvests the PERSISTENCE of crisis (avoiding the back half + the choppy recovery), NOT the onset.
• Real-time instrumentation. A daily cron computes RV_t, VIX close, current state, days-in-state, distance-to-next-threshold, and RV-vs-VIX divergence; logs to the state file; and surfaces a channel message ONLY when the multiplier would change. The "distance to next threshold" telemetry ("RV at 19.2%, 0.8pp from STRESSED") is the early-warning the founder sees before a flip. Verified by: observability test asserting the daily cron emits all of {state, days-in-state, distance-to-threshold, RV-VIX divergence}; AND a measured-lag report in the wet rehearsal that quantifies onset-detection lag per historical stress episode (so the lag claim is data-backed, not asserted).

Wet-Rehearsal (System Validation) Plan

The load-bearing section. Borrowed from the launch industry: a wet rehearsal is a full dry-run of the entire chain on historical + paper data to prove it's any good BEFORE turning it on. Go/no-go bars are pre-registered before looking at out-of-sample results (so the bar is not fitted to the answer). This plugs directly into the existing [[investing-backtest-thesis]] harness and the V-model right-arm system-validation gate.

Window shape

Anchored walk-forward. Freeze thresholds + hysteresis on the pre-2010 calibration block. Then roll: estimate the matrix using only data up to T, generate state + multiplier for [T, T+3mo], step forward, re-estimate, never peek. Leave-one-out by stress episode per house doctrine: drop 2008, refit, does the overlay still help in 2020? The result must not rest on a single crisis.

The three bars (ALL must pass to wire it to sizing)

Bar 1 — State persistence / random-walk disproof (run FIRST, gating). Diagonal self-transition probabilities materially above base rate (e.g. P(CALM->CALM), P(NORMAL->NORMAL) > ~0.9 day-over-day) AND the Markov-vs-iid LR test rejects iid at p<0.01 AND dwell-time is right-shifted vs the geometric null (the grafted dwell-time test). Expected to PASS strongly (vol clustering is robust); its value is being the explicit mechanical disproof of the #1 failure mode + a smoke test that the labeler isn't broken. Pairs to: FR-3.

Bar 2 — Calibration (the honesty check). Predicted forward-return / drawdown distribution conditioned on state must match realized base rates out-of-sample, within CI, across the walk-forward. STRESSED-entry realized drawdowns must actually exceed CALM-entry drawdowns. Pairs to: FR-5.

Bar 3 — Risk-adjusted improvement AFTER costs (the does-it-pay test). Compare the vol-overlaid book vs the un-overlaid buy-and-hold book (the honest benchmark — buy-and-hold is what beat the v1.1 mechanical exits). To turn ON: materially higher Sharpe AND materially lower max drawdown, NET of estimated transaction costs (maker-limit + slippage), WITHOUT giving back so much total return that it loses to buy-and-hold on absolute terms by more than a small margin — across >=2 leave-one-out stress episodes — with the benefit coming from the SPY+satellite sleeve, NOT from selling the conviction core at the bottom. Pairs to: FR-6, FR-7, FR-8.

The honest expected outcome: bars 1+2 pass, bar 3 is the one most likely to FAIL (vol is the most-watched, most-priced variable in markets; knowing it stays high tomorrow is not proprietary, and the trailing-window lag means we de-risk near the low / re-risk after the bounce). If 1+2 pass but 3 fails: keep the state estimator running as a real-time founder-facing risk DASHBOARD and do NOT wire it to sizing. That is a clean, generalizable finding, not a failure.

Paper-trade rehearsal stage (founder-gated)

Only AFTER all three bars pass on historical walk-forward: wire calibrated multiplier -> quarter-Kelly sizing -> Alpaca paper maker/limit DAY orders. Run live-paper >=2 weeks (longer if no regime change occurs in the window), comparing live-emitted states + multipliers vs what the backtest would have produced on the same days, and monitoring Bar-2 calibration LIVE (the transition matrix may be non-stationary post-2010; calibration must be watched in deployment, not just at backtest time). Paper-trade deploy requires explicit founder authorization per [[feedback_paper_trade_deploy_authorization]] (canonical DECISION-format APPROVE or explicit "deploy" verb — structural agreement is not enough).

Traceability stub

Open questions for founder

1. History backfill scope. Do we invest in the pre-1993 S&P 500 daily splice to roughly double the CRISIS/STRESSED sample, or accept the thinner 1993-present window and lean harder on the CRISIS shrinkage prior? The splice materially strengthens Bar 1/Bar 2 for the rare states but adds a data-stitching surface (index vs ETF, dividend treatment).
2. Threshold philosophy. Fixed-frozen absolute thresholds (chosen: transparent, economically interpretable, but vulnerable to structural vol-regime drift) vs rolling-percentile thresholds (adapt to a changed vol environment, but reintroduce lookahead/drift). We chose fixed+frozen and monitor drift via the VIX cross-check — is that the right call, or do you want a percentile variant tested as a secondary?
3. Multiplier schedule values. The 1.0/1.0/0.6/0.35 CALM/NORMAL/STRESSED/CRISIS schedule is a first-cut hyperparameter. Do you want it fixed by hand (transparent) or tuned on the calibration block (risks overfitting the throttle to historical crises)?
4. Dashboard-first vs sizing-first ambition. Given the honest expectation that Bar 3 may fail, are we comfortable funding this primarily as a regime dashboard with sizing as the stretch goal — or is a dashboard-only outcome not worth the build cost to you?
5. Secondary direct-vol mode. Keep the short-vol-carry diagnostic (paper-only, to measure how badly transition-lag hurts) in scope for the prototype, or cut it entirely to keep the first build tight?

Related

• [[2026-05-27-markov-equities-pipeline-spec]] — the parent pipeline spec (states / transition matrix / Monte-Carlo / calibration / quarter-Kelly / maker-limit machinery this inherits)
• [[2026-05-18-memory-cycle-v1.1]] — the conviction book the overlay ring-fences; source of the no-mechanical-exits doctrine
• [[2026-05-29-v-model-build-workflow-spec]] — the V-model build discipline this run follows (spec-test pairing + traceability spine)
• [[investing-backtest-thesis]] — the walk-forward, survivorship-free, leave-one-out validation harness the wet rehearsal plugs into
• [[feedback_alpaca_paper_opg_unreliable]] — why FR-7 mandates fractional DAY limit orders, never OPG/market-on-open
• [[feedback_paper_trade_deploy_authorization]] — the paper-trade deploy gate the rehearsal stage respects