Markov-Chain Equities Pipeline — build spec (v0, pending founder)

Origin: founder shared a Polymarket Markov framework 2026-05-27 ([[2026-05-27-polymarket-markov-quant-framework]]), then: "No prediction markets, just stocks. Markov chains have always fascinated me but I've never put one into production. Curious about getting this pipeline built out." This is a capability + learning build with honestly-measured, modest edge expectations — not a money-printer. Paper-traded on the existing Alpaca scaffold; live trading off the table without explicit founder authorization.

⭐ UPDATE 2026-05-27 (founder direction) — capital-cycle state machine, NOT generic regimes

Founder redirected the formulation (and it's better than my v0 options): model the chip-fab / semiconductor capital cycle as the explicit state machine. "We are not day traders. Our most-developed theory is chip-fab capital cycles. We're in Phase 2 (capacity announcements), not yet Phase 3 (capacity online). That's a clear state machine — which phase are we in, what's the probability of moving to the next." Plus: discretized price buckets at multiple horizons as a complementary lens.

This is the anchored version — and it plugs into infra RDCO already has:

• The cycle = our existing [[2026-05-18-memory-cycle-v1.1]] thesis (DRAM/NAND is the canonical capital-cycle commodity). Not greenfield — the domain content + conviction already exist.
• Phase labeling already exists: /investing:label-historical-phases skill already labels historical cycle phases → it's the transition-observation generator for the phase-tracker.
• Walk-forward harness already exists: /investing:backtest-thesis.

Two-layer architecture (supersedes the v0 "design decision" below):

1. Capital-cycle phase tracker (macro, strategic — sets direction + conviction). States = cycle phases: P1 tight-supply/rising-ASP → P2 capacity-announcements (←here) → P3 capacity-online/glut → P4 capex-cuts/rationalization → back to P1. Honest wrinkle: only ~4-6 semi cycles in 40 yrs = too few to fit a transition matrix. So this layer = explicit phase definition + observable leading indicators (capex announcements, fab groundbreaks, fab-utilization, lead times, book-to-bill, inventory days, ASP trend) + informed transition priors, fed by /investing:label-historical-phases. It answers "what inning are we in + is the next-phase signal firing."
2. Price-bucket multi-horizon Markov (tactical, data-rich — times entries). The actual transition-matrix + Monte-Carlo math from the source framework, run on chip-exposed tickers at weekly/monthly horizons (NOT daily — not day-traders). Times entries inside whichever phase Layer 1 says we're in.

Net: Layer 1 sets the bet (long the cycle through P1-P2, de-risk before the P2→P3 turn where the cycle kills people); Layer 2 sizes + times within it. Both paper-first, walk-forward. The two-layer design below (states/MC/calibrate/Kelly/limit-execution) still applies — it's Layer 2's machinery + the sizing/execution for both.

The core translation problem (why Polymarket→stocks isn't 1:1)

On Polymarket, price = probability (0-100¢ = P(YES)) with a fixed binary resolution, so a Markov chain over price-states maps directly to "P(YES at expiry)." Stocks have no bounded price, no binary outcome, no fixed expiry. So three things must be redefined:

The design decision (recommend: regime-state, not price/return-level)

• Daily stock returns ≈ random walk at the price/return level — a return-bucket Markov chain will likely find thin-to-no edge (efficient-market headwind). Building that first mostly teaches us it doesn't work.
• Regimes persist — volatility clusters (GARCH-style), trends have momentum, mean-reversion regimes exist. A Markov chain over regime states has a genuine shot because the memory is real.
• Recommendation: build a regime-state Markov model. States = a small grid like {trend-up, range, trend-down} × {low-vol, high-vol} (≈6 states), derived from rolling indicators (e.g. MA-slope sign + realized-vol bucket). Optionally evolve to a Hidden Markov Model (HMM) later where the regime is latent and inferred. Start with observable-regime states (simpler, debuggable) before HMM.

Pipeline (5 stages, equities-adapted)

1. State definition — map each historical bar to a regime state via transparent rules (MA slope + realized-vol bucket). Keep it interpretable.
2. Transition matrix — count regime→regime transitions over a long, multi-regime history; enforce ≥30 observations per state (the discipline the source article preached then violated). Per-ticker AND/OR pooled across a universe for robustness.
3. Monte Carlo — from current regime, simulate forward paths → distribution of N-day forward returns. (Map regime-sequence → return draws via each regime's empirical return distribution.)
4. Calibrate — compare model's predicted forward-return probabilities against realized historical base rates for that regime. If the model says "60% up" it must actually happen ~60% of the time. This is the load-bearing honesty check.
5. Size + execute — quarter-Kelly position sizing on the calibrated edge; limit/DAY orders on Alpaca paper (maker discipline + the known [[feedback_alpaca_paper_opg_unreliable]] lesson: fractional DAY notional, not market-on-open).

Validation (non-negotiable, = our existing doctrine)

Per [[investing-backtest-thesis]]: walk-forward (re-estimate the transition matrix using only data available at each point — zero lookahead), survivorship-free universe (construct from point-in-time membership), out-of-sample test windows, leave-one-out reporting, honest confidence intervals (no 2-decimal Sharpe on tiny trade counts). A model that's brilliant with lookahead bias bleeds out live.

Phasing

• Phase 1 (engine + backtest, NO execution): regime-labeler → transition matrix → Monte Carlo → calibration report, walk-forward on a handful of liquid tickers (e.g. SPY + a few large-caps). Output: is it calibrated? Is there any edge after costs? Decision gate: if calibration fails or edge is nil, we learned the EMH lesson cheaply and stop or pivot to HMM.
• Phase 2 (paper-trade, founder-gated): wire calibrated signals → quarter-Kelly sizing → Alpaca paper limit orders. Run live-paper for ≥2 weeks; compare calibrated probabilities vs actual outcomes.
• Phase 3 (scale/iterate): only if Phase 2 calibration holds — widen universe, consider HMM latent-regime upgrade.

First increment (reversible prep, buildable now on greenlight)

Phase 1 engine on historical data only — pure research/backtest, no Alpaca connection, no execution. Build as a todo-file + dynamic /loop sequenced build (per [[feedback_todo_file_loop_vs_notion_queue]]) with a running implementation-notes file (per [[feedback_implementation_notes_sub_agent_pattern]] — financial-execution-adjacent → decisions/deviations/open-questions logged). PR-only on the repo ([[feedback_pr_only_workflow]]).

Gates / guardrails

• No live trading without explicit founder authorization (canonical DECISION-format or explicit "deploy" verb per [[feedback_paper_trade_deploy_authorization]]).
• Paper-trade deploy = also founder-gated; Phase 1 (backtest only) is the un-gated reversible part.
• Implementation-notes file required for the build.

Open design questions for founder

Answered by founder 2026-05-27:

• State formulation → capital-cycle phase states (Layer 1) + multi-horizon price buckets (Layer 2). Not generic regimes, not HMM-first.
• Horizon → weekly/monthly, NOT daily (not day-traders).

Still open:

1. Layer-1 phase definitions: adopt the phases as already framed in [[2026-05-18-memory-cycle-v1.1]] verbatim, or re-derive a crisp 4-phase definition + the indicator thresholds that trigger each transition? (I'd reconcile the two and bring you one canonical phase ladder.)
2. Universe: the chip-exposed names — pure-play memory (MU + the SK Hynix / Samsung ADRs), or broader semi-cap-equipment + foundry (AMAT/LRCX/KLAC/TSM)? Different names sit at different points in the same cycle (equipment leads, foundry lags).
3. Repo: new markov-cycle repo, or fold into the existing Alpaca/autoinv scaffold?

(Phase-1 engine build doesn't block on these — they shape Layer-1 + Phase-2 paper-trading.)

Related

• [[2026-05-27-polymarket-markov-quant-framework]] — the source article + honest assessment
• [[investing-backtest-thesis]] — the walk-forward / survivorship-free harness this plugs into
• [[feedback_paper_trade_deploy_authorization]] / [[feedback_alpaca_paper_opg_unreliable]] — execution gates + lessons
• [[feedback_implementation_notes_sub_agent_pattern]] / [[feedback_todo_file_loop_vs_notion_queue]] — build mechanics