CCA-Foundations Practice Exam 2 — Miss Analysis

Score: 42/60 = 70% · finished with 12 min to spare in the 2 hr window. Exam 1 (cold, 2026-06-14): 50/60 = 83%. Pass line: recorded as 72% in cert-progress (source unverified — CONFIRM the real cut score). If 72%, exam 2 lands ~2 pts under; if 70%, it's a pass on the line. Coverage: all 18 misses captured (D1×7, D2×3, D3×4, D4×2, D5×2).

Headline: the drop is targeting, not regression

Exam 1's misses were spread (harness mechanics, plausible-neighbor definitions, BEST-answer traps); D4 was a clean sweep. Exam 2 concentrates the misses into a few nameable, cheap-to-close areas + one judgment axis + a context-economy thread. This isn't a skills decline — exam 2 probed the seams exam 1 happened to miss. Every pattern is closable without heavy study; the single biggest one is pure test-craft.

Four patterns, ranked by ROI to fix

1. "Most-complete answer" under-selection — TEST-CRAFT · ~5 Qs · highest ROI, zero studying

Q4 (D3), Q5 (D3), Q19 (D2), Q56 (D3), Q18 (D5), and the flavor in Q23. On any "which is BEST / most robust / most reliable / MOST addresses" stem, the keyed answer is almost always the comprehensive, layered, both-halves, one-notch-further option. Founder repeatedly picked a clean single-mechanism answer that was correct-but-incomplete:

• Q4 — picked "example alone"; key = "spec plus example."
• Q5 — picked "Write hook + .claudeignore"; key = ".claudeignore + Write hook + Bash hook + OS read-only" (defense in depth).
• Q19 — picked the "reasonable but incomplete" audit-log option; key adds response-outcome logging + append-only storage + access controls.
• Q56 — picked "read the research log"; key = "read and append after" (accumulate, not just look up).
• Q18 — picked "proportional stratified sampling"; key = "stratified with minimum quota per stratum" (proportional still undersamples the rare 10% regulatory-filing stratum).

Fix (a rule, not study): when the stem says BEST/most-robust/most-reliable, choose the most thorough option that combines mechanisms. ~8% of the exam sits here.

2. Hook scoping / mechanics — KNOWLEDGE · ~3 Qs · recurring weak area

Q42 (D1), Q5 (D3), Q23 (D3). The rule that keeps biting:

• Tool-specific hooks only intercept that tool. A PreToolUse hook on Write does NOT catch Bash shell-redirection writes (echo content > file). Comprehensive file protection needs a Write hook and a Bash hook (+ .claudeignore + OS perms).
• Exit-code convention: 0 = allow, non-zero = block (Q42 — founder flipped it). This was already Cluster A on exam 1. It's persisting → re-read [[2026-06-14-cca-harness-mechanics-cheatsheet]].

3. API surface — KNOWLEDGE · ~3 Qs · the long-flagged D3-API gap, now confirmed

Q7 (D4), Q33 (D4), Q52 (D2). Exactly the "D3 API specifics" weak area flagged 2026-06-10 (tool_choice was named then):

• tool_choice to FORCE a tool call (fixes "responds in prose instead of calling the tool"); pair with downstream validation for field-value correctness (Q7). JSON mode is not the mechanism for forcing a call.
• Prefilling { locks JSON from the first token — beats "Continue your previous response" overrides and beats merely-stronger instructions (Q33).
• MCP capability-change notifications are protocol-level and transport-agnostic (STDIO servers can notify too); cache discovered capabilities + refresh on notification is the standard multi-server optimization (Q52).

Format-control hierarchy (the connective insight — Q7 + Q33 are one stack): for JSON output, strongest → weakest: (1) forced tool_choice with your schema as the tool's input_schema → gives schema validation, the production answer (Q7); (2) prefill { → token-level format lock, but only guarantees the first char, no schema check, incompatible with extended thinking, and the prefilled { isn't echoed so you prepend it when parsing (Q33); (3) stronger instructions → weakest, override-able. Founder's "prefill feels hacky" read is the correct production instinct — prefill is a legit documented technique but a narrow token-level fallback; reach for tool_choice/structured outputs for guaranteed JSON. He missed both Q7 and Q33, i.e. the whole stack — that's the real lesson, not either trick alone.

4. Act-vs-ask calibration — JUDGMENT (not knowledge) · ~4 Qs

Q8, Q15, Q17, Q55 (all D1). Miscalibrated autonomy threshold, and the misses cut both ways:

• Too quick to ASK the user: Q8 (two valid divergent outputs, neither specified → present both if affordable, only ask if costly), Q55 (additive 4th database → accommodate + note it, don't stop-and-ask).
• Too quick to ACT autonomously: Q15 (47 vs 10-15 modules = material resource jump → surface scope + options), Q17 (autonomous schema change without goal-constraint check = a design RISK, not "appropriate dynamic planning").

The rubric to internalize:

(This is the same act-vs-ask dial Ray tunes with the founder daily — calibration, not a content gap.)

5. Context economy / right-sized output — KNOWLEDGE-JUDGMENT · ~2 Qs (D5 + D1 thread)

Q46 (D5), reinforced by Q9 (D1). Match information granularity to the consumer's decision need:

• Q46 — subagent classified 500 docs; founder picked "return all 500 rows." Key = "return counts-per-class + top 5 for prioritization." The coordinator's task (pick which classes to process) needs class-level summary, not row-level detail — and returning a file path just defers the same context cost to read-time.
• Q9 — across sessions, the keyed answer is progressive summarization (save structured summaries, load those, not raw docs) for the same reason: bound the context to what the next step actually needs. Theme: don't over-return. This is a context-management instinct, adjacent to the act-vs-ask calibration above.

Founder's "bad questions" instinct — partly validated

• Q23 has a duplicate-answer artifact (two options describe the Bash-redirect cause) — but his actual pick (exit-code reversed) was still genuinely wrong.
• Q56 is a hair-split between two near-equivalent correct answers; the real differentiator is the append-to-accumulate step.
• Q9 "conflict detection" distractor is plausible but doesn't address the PRIMARY challenge (context capacity → progressive summarization is the keyed pattern). Net: a couple are genuinely soft, but each still hinges on a real distinction worth banking.

Study plan (ROI order)

1. Adopt the "most-complete answer" heuristic (test-craft, ~5 Qs) — biggest, cheapest lift.
2. Re-read the hook cheatsheet; lock "Write hook ≠ Bash writes" + exit-code convention (~3 Qs).
3. API-surface drill — tool_choice, prefilling, JSON-mode vs tool-use, MCP notifications protocol-level (~3 Qs). This is the /cert-study D3-API drill queued since 6/10.
4. Act-vs-ask rubric — internalize the 4-row table (judgment, ~4 Qs).
5. Two discrete D5 facts — min-quota stratified sampling (for rare strata) + match subagent/response granularity to the consumer's decision (don't over-return) (~2 Qs).

Closing patterns 1–3 alone (~11 Qs of headroom) clears the pass line comfortably, regardless of whether the cut is 70% or 72%.