06-reference/research

dsr cross block accounting after reseal

2026-10-29·research-brief·source: deep-research·by Ray Data Co (deep-research synthesis)
deflated-sharpeeffective-nmultiple-testingsealed-holdoutpre-registrationautomated-investing

There is no such thing as a correlation-discounted increment: count the retest, recompute N_eff on the pooled ledger, and forbid Eq. (9)

The question

"Decide the cross-block DSR accounting rule after a sealed-holdout reseal: does a candidate re-tested on a later block count as N+1 trials, or does a correlation-discounted increment apply because consecutive blocks share a regime?"

Acronyms used throughout, expanded once here: DSR = Deflated Sharpe Ratio (Bailey and López de Prado 2014). CPCV = Combinatorial Purged Cross-Validation. ONC = Optimal Number of Clusters, the unsupervised clustering algorithm of López de Prado and Lewis. MinTRL = Minimum Track Record Length. ENB = Effective Number of Bets. FWER = familywise error rate. EVT = Extreme Value Theory. N_eff is the correlation-adjusted effective number of trials; N_literal is the raw ledger row count.

This is open-follow-up #2 of [[2026-08-31-sealed-holdout-refresh-policy]], which pre-registered roll-forward reseal and left the cross-block invariant stated but unquantified ("cumulative N spans blocks... or a reseal silently launders the multiple-testing discount"). It composes directly with the estimator pre-registered in [[2026-08-31-dsr-effective-n-estimator-precommit]]. It is a design decision in our pipeline, not a literature summary, so it ends in a rule precise enough to code and pre-register.

What we already know (from the vault)

What the web says

Convergences and contradictions

Synthesis for RDCO

The honest answer is the less clever one, and the arithmetic says the clever one buys nothing anyway. Count the retest. N_literal += 1, naive, unconditional. The correlation adjustment is applied exactly once, inside the N_eff clustering that [[2026-08-31-dsr-effective-n-estimator-precommit]] already pre-registered, and it is applied by recomputing N_eff on the pooled cross-block ledger rather than by adding a discounted increment to a running total. Three independent reasons converge on this, and they do not depend on each other. First, the primary's own condition: estimating an average correlation requires T ≥ ½M(M−1). At the ~1,005-bar validation partition that caps the trial matrix at M ≤ 45 trials, and at the ~300-bar block scale at M ≤ 25 (my arithmetic from the primary's condition); even a full-history SPY cache around 8,000 bars caps it at M ≤ 126. v0's sweep counts grid cells in the thousands. The paper does not stop there, and neither should this brief: p.15 offers two escapes from its own condition, and the reason neither rescues Eq. (9) for us is an argument I owe, not a bar the paper sets. The first escape, reduce the dimension of the correlation matrix and compute the average correlation on the reduced definite-positive matrix, does not give us a second discount: collapsing M correlated trials into a smaller set of representatives is the same operation ONC already performs under the companion pre-registration, so taking the reduced-matrix ρ̂ and also clustering would double-count the identical structure. It converges on our rule rather than competing with it. The second escape, information-theoretic estimation of N̂ via total correlation or multiinformation, replaces a second-moment estimator with an entropy estimator over the same M variables and the same T observations; entropy estimation over high-dimensional continuous variables is more sample-hungry than correlation estimation, not less, so at a T where the correlation is already "pointless" by the paper's own test, the information-theoretic route is worse off, not better. That is standard estimation reasoning and it is mine; I did not read Watanabe [1960] or Studený and Vejnarová [1999] and I am not citing them for it. The conclusion survives the correction: the correlation route is unavailable to us at v0's trial counts, by the paper's condition and because its own two remedies do not reach our regime. Second, Halle et al.'s measured non-additivity: summing block-wise effective-N under-counted the family by 44% in a case with no true cross-block dependence, and only the undiscounted Bonferroni and Šidák corrections controlled FWER across all their scenarios. Third, numerical immateriality, below.

State the direction of the error plainly, because this is the self-serving fork. A discounted increment lowers N, which lowers SR₀, which lowers the hurdle, which makes GATE-1 easier to pass. It is the permissive direction, and it is the direction a pipeline operator wants when a candidate is sitting just under the bar. The correlation-discount route would absolutely flip marginal candidates, and the primary's own worked example is the proof. Bailey and López de Prado's example (T=1250, V[{SR_n}]=1/500, γ̂₃=−3, γ̂₄=10, annualized SR 2.5) prints DSR = 0.9004 at N=100, a reject at 95%, and notes that at N=46 it would have been 0.9505, a pass (pp.9-10; both constants reproduce to four decimals in my own computation). Applying Eq. (9) to that same M=100 ledger, ρ̂ = 0.5408 yields N̂ = 46.5 and DSR = 0.9500 — the paper's own rejected strategy becomes a funded one, with no new data, purely from an estimated scalar. A plausible-sounding average correlation of 0.54 among grid-neighbor trials is not a stretch; it is what you would expect. That is the whole risk in one number. By contrast the naive cross-block increment is nearly free: in the same example, moving N from 100 to 101 changes DSR by 0.0008. Across ledger sizes, the gap between N+1 and a half-discounted N+0.5 is 3.9% of SR₀ at M=5, 0.55% at M=20, 0.069% at M=100, and 0.002% at M=2,000 (my computation of the SR₀ formula). The conservative choice costs essentially nothing at any realistic ledger size, and the permissive choice is worth a whole confidence level. There is no trade here worth making.

Can we estimate ρ from our own data? No, and the parent brief's finding generalizes. The cross-block estimand is not even the primary's ρ. It is the correlation, across candidates, between a candidate's block-b Sharpe and its block-(b+1) Sharpe. Estimating it requires at least two realized sealed blocks and a population of candidates tested on both. We have one cached series of 1,509 bars, a queryable-block floor of 1,000 bars under the parent policy, and a minimum reseal accrual of 252 new bars; block 2 does not exist and cannot exist for years. Even granting it: the sample of candidate pairs is itself correlated (grid neighbors), so the effective sample size for ρ̂ is the cluster count, not the candidate count. Under Fisher's z-transform, which the DSR primary itself names as the fix for ρ̂'s estimation error, a true ρ̂ = 0.5 at n = 20 effectively-independent pairs carries a 95% interval of [0.07, 0.77], so the implied increment (1 − ρ̂) ranges over [0.23, 0.93] — a four-fold range on the quantity doing the discounting. At n = 10 the interval includes zero. This is the same finding the parent brief reached about the holdout Sharpe itself: estimator noise, not leakage, is what binds. A rule requiring a well-estimated ρ is unimplementable at RDCO's data scale, and that is the finding, not a deferral.

Composition with the companion estimator: one discount, in one place, with a ratchet on the hurdle itself. The cross-block correlation adjustment and the within-block clustering adjustment are the same adjustment if, and only if, N_eff is recomputed from scratch on the pooled ledger rather than accumulated. When a block-2 sweep rediscovers a block-1 candidate, ONC will place the two rows in one cluster and the pair will contribute roughly 1 to N_eff rather than 2. That is the discount, applied once. Adding N += (1 − ρ̂) on top is the double-discount the question warns about. There is a second, subtler double-count to avoid in V[{SR_n}]: for M equicorrelated trials the expected sample cross-sectional variance is σ²(1 − ρ), not σ² (standard result, derived here), so computing V[{SR_n}] across all correlated trial Sharpes already shrinks the dispersion by (1 − ρ) and shrinks SR₀ by √(1 − ρ) — a 29% lower hurdle at ρ = 0.5, entirely silently. López de Prado and Lewis §8.2 pre-empts this by taking the variance across the K cluster-representative series, which is what the companion brief's step 5 already specifies. That step is now primary-confirmed, including its aggregation rule (minimum-variance weights, annualize per cluster frequency, express at the selected strategy's frequency), closing the companion's open follow-up #1 (the aggregation rule is what the primary confirms; the cross-block reading of it is mine). Keep it; do not substitute the all-trials dispersion, which is the permissive one. And close the loop, because naming the variance channel is not the same as guarding it: since SR₀ is linear in √V[{SR_k}], every one of these dispersion effects is a hurdle effect, and an anti-laundering rule that polices only the trial count cannot see them. A 0.14% drop in the cluster-Sharpe dispersion at N_eff = 100 cancels a whole extra trial; 1.1% does it at N_eff = 20 (my arithmetic). That is why R4 below asserts monotonicity on SR₀ and not on N_eff, which is the single most important change between this rule and the obvious version of it.

PRE-REGISTERED RULE — CROSS-BLOCK TRIAL ACCOUNTING (CBTA-1). This is the thing to code.

A candidate re-tested on a later block counts as a full additional trial in the ledger. No correlation-discounted increment is ever applied. The correlation adjustment happens once, by recomputing N_eff on the pooled cross-block ledger.

R1 — Ledger counter. Every evaluation of a candidate against a new block's validation partition writes a new ledger row: N_literal += 1, unconditionally, regardless of similarity to any prior row. The row records block_id, prior_trial_id (the earlier row it re-tests, or null), and retest_class ∈ {NEW, REDISCOVERY, RECONFIRM}, assigned before the trial is scored.

R2 — One discount, one place. N_eff is recomputed from scratch on the pooled cross-block ledger per [[2026-08-31-dsr-effective-n-estimator-precommit]] steps 2-4. It is never accumulated from per-block estimates, and no ρ term appears anywhere else in the pipeline. Rationale is citable, not stylistic: block-wise effective-N is not additive (Halle et al. 2016).

R3 — Pinned clustering window (anti-laundering). The correlation matrix C fed to ONC is computed on a single pinned reference window W_ref, with every surviving ledger trial's return series re-evaluated on W_ref (cheap: re-run the param vector through the vectorbt sweep). W_ref is the current block epoch's TRAIN/VALIDATION partition — about 1,005 bars today and growing with each reseal — explicitly NOT the ~300-bar sealed holdout block, which stays import-walled out of this object entirely. Using each trial's own full history instead would let shared pre-block data manufacture correlation and silently shrink N_eff. A trial that cannot be re-evaluated on W_ref keeps its ledger row, is excluded from the clustering, and counts as its own cluster (+1 to N_eff).

R4 — Monotonicity invariant, asserted on the hurdle and not on the count. The quantity that must never fall across a reseal is SR₀, not N_eff. SR₀ = √V[{SR_k}] · ((1−γ)Φ⁻¹(1−1/N_eff) + γΦ⁻¹(1−1/(N_eff·e))) is linear in the cluster-Sharpe dispersion, so a monotone N_eff does not imply a monotone hurdle: at N_eff = 100 a 0.14% fall in √V[{SR_k}] exactly cancels N_eff += 1, and at N_eff = 20 it takes 1.1% (my arithmetic from the SR₀ formula). An accounting that polices only the count leaves the whole variance channel open, in the permissive direction. Therefore: SR₀_gate = max(SR₀_new, SR₀_prev), compared in the companion's pinned units (per-return-observation, same bar frequency, same cost model). Log N_eff, N_eff_prev, V[{SR_k}], V_prev, SR₀_new, SR₀_prev, and SR₀_gate on every post-reseal run.

R4a — Resolution of the R2/R3/R4 conflict, pre-registered rather than left to the implementer. R2 recomputes N_eff from scratch and R3 pins a new W_ref each block, so neither carries a monotonicity guarantee and a legitimate run can produce SR₀_new < SR₀_prev. R4 must therefore clamp and flag, not halt: take SR₀_gate = max(SR₀_new, SR₀_prev), mark the run HURDLE_RATCHET_APPLIED, and surface it for review. A hard halt would create pressure to reuse the previous window, which silently violates R3 and is exactly the convenient wrong answer. Failing conservative beats failing closed here.

R5 — RECONFIRM carve-out (N_eff contribution = 0). Re-testing an already-DEPLOYED, previously-PASSED candidate on a new block writes N_literal += 1 for audit but is excluded from the GATE-1 family only if all four hold: (a) the bar was pre-registered before the block sealed; (b) no parameter was re-tuned, at all; (c) the test is two-sided with a pre-committed DE-DEPLOY action on fail, so it is not a free option; (d) the row cites the original pass's trial_id. Any violation reclassifies the row to REDISCOVERY and it re-enters the family. Rationale: a single pre-specified hypothesis with a symmetric consequence is not a selection among alternatives, so it contributes no selection bias to the discovery family. This is derived, not cited — medium confidence, and the four conditions are what make it defensible. Direction of error, stated because it is the exception: R5 is the only clause in CBTA-1 that lowers a hurdle, so it is the only one that can produce a false positive. The four conditions are conjunctive precisely to bound it, and (b) and (c) are the load-bearing pair — a re-tuned parameter makes it a selection, and a fail with no de-deploy makes it a free option. If a reviewer has to argue about whether a condition is met, it is not met.

R6 — Resurrection stays forbidden. [[2026-08-31-sealed-holdout-refresh-policy]] permanently retires a FAILED candidate and its param neighborhood. CBTA-1 does not relax that. If it is ever relaxed, the resurrected candidate is excluded from cluster-collapse and floors N_eff at +1 with no discount, because a second attempt at the same gate is pure alpha-spending.

R7 — V[{SR_k}] stays on cluster representatives. Per López de Prado and Lewis §8.2: min-variance-weighted cluster return series, each annualized at its own Frequency_k, expressed at the selected strategy's frequency. Log the all-trials dispersion alongside it for audit, flagged not-for-use. If the cluster-representative value is ever the smaller of the two, flag the run for review rather than proceeding.

R8 — Eq. (9) is forbidden, with a visible reason. No average-correlation discount may enter SR₀. Reinstating it requires T ≥ ½M(M−1) on the clustering window, per the primary (p.15). The ledger reports M_max = floor((1+√(1+8·T_ref))/2) next to N_eff on every run, computed on W_ref's length, so the violation is visible in the artifact rather than argued about later.

R9 — ONC gets NO exemption from the condition used to forbid Eq. (9), and this is an amendment to the companion's refusal clause. It would be incoherent to forbid the average-correlation route on T < ½M(M−1) while feeding the same MxM matrix to ONC unguarded. The failure mode is real and permissive: with M > T the sample correlation matrix has rank at most T, so spurious shared structure appears, clusters over-collapse, K_onc falls, N_eff falls, and the hurdle falls. [[2026-08-31-dsr-effective-n-estimator-precommit]] already refuses and falls back to N_eff = N_literal when the trial matrix is "degenerate"; that word is hereby given a number. Degenerate means M > M_max(T_ref). When it holds, the estimator refuses: N_eff = N_literal, no discount, run flagged CLUSTERING_REFUSED_ILLCONDITIONED. The stricter ½M(M−1) bar is used rather than the looser rank bar M ≤ T because more refusals means a higher hurdle, which is the direction this gate is supposed to err in.

What R9 actually costs, said plainly rather than buried. At T_ref ≈ 1,005 bars, M_max = 45. v0's sweep counts grid cells in the thousands, so under CBTA-1 as written the clustering discount will refuse and N_eff will equal N_literal on every realistic v0 run. The N_eff machinery is not dead, but it is dormant until one of two things changes: the sweep keeps M under M_max (which is v0's original blunt mitigation, "cap grid granularity so cells are meaningfully distinct," arrived at here from the opposite direction and now with a number attached), or the ledger is dimension-reduced before clustering per the primary's own first remedy. Either is a legitimate way to earn the discount. Neither is a reason to skip the guard.

Report per run: N_literal, N_eff, N_eff_prev, the three cluster estimates, the per-block row counts by retest_class, W_ref start/end and T_ref, the count of trials excluded from clustering under R3, M, M_max, V[{SR_k}] both ways plus V_prev, SR₀_new, SR₀_prev, SR₀_gate, and the HURDLE_RATCHET_APPLIED / CLUSTERING_REFUSED_ILLCONDITIONED flags, alongside SR_obs and T.

Test and implementation plan, so this is checkable rather than asserted. (1) Unit test R4 on the hurdle, not the count: construct a two-block synthetic ledger in which N_eff rises by 1 while the cluster-Sharpe dispersion falls by 0.2% at N_eff ≈ 100, confirm SR₀_new < SR₀_prev (the laundering channel a count-only invariant would miss), and assert the ratchet fires and sets HURDLE_RATCHET_APPLIED. Separately assert the pooled-history clustering window fails loudly against R3. (1a) Unit test R9: feed a matrix with M > M_max(T_ref) and assert N_eff == N_literal with the refusal flag set, rather than a silently-collapsed K_onc. (2) Unit test R3's artifact: generate one strategy evaluated on a short window and the same params on a time-superset window, confirm their measured correlation on the pooled-history construction exceeds their correlation on W_ref, which is the mechanism R3 defends against. (3) Regression-test the DSR implementation against the primary's printed constants before anything else is trusted: N=100 must give 0.9004 and N=46 must give 0.9505 on the paper's inputs. (4) Extend v0's pure-noise 100-cell V-model test across a synthetic reseal and assert the false-positive rate does not rise after the second block. (5) Log retest_class from the first block, before any reseal exists, so the classification is never retrofitted.

Why this is in the vault

This closes open-follow-up #2 of [[2026-08-31-sealed-holdout-refresh-policy]] with a rule that can be coded as written, and it is a Phase-2 build input to two specific unbuilt objects in /Users/ray/Projects/automated-investing: the HoldoutManager reseal path and the multiple-testing-ledger schema behind validation.py: deflated_sharpe(...). It also closes open-follow-up #1 of [[2026-08-31-dsr-effective-n-estimator-precommit]] as a byproduct, by reaching the López de Prado and Lewis primary (§8.2, minimum-variance cluster aggregation) that two prior attempts failed to extract. Concretely, it prevents the specific failure where the first reseal silently launders the trial discount and every DSR the pipeline reports afterward is too permissive.

Open follow-ups

Related

Sources

Vault:

Web (all three primaries below were deep-read from retrieved full text, not search extracts):

Computed in this brief (my own arithmetic, from the primaries' published formulas and inputs; reproducible from the formulas above):