What progress have you made since your last update?

This update is late: Phase 2 finished on August 11 and a correction followed on August 21. Both are reported here for the first time.

The Phase 2 design posted on July 18 ran to completion: 22,140 judgments, 100% complete, $172.86 against a $173.80 projection. The original analysis was pinned before unblinding and run once.

Result. In our three fictional worlds, giving the judge a two-call oracle budget during sequential debate, versus the same debate with no oracle, increased judge error. Under the label assignment actually run, the estimate is +3.9 percentage points (95% CI [2.2, 5.7], Holm p = 0.0006).

Erratum (August 21). The Phase 3 canary exposed a harness defect: the protocol's mirrored-sides replicate was never implemented, so both replicates of every judgment showed the correct answer in the same position, and the side-bias gate as computed was non-diagnostic. We withdraw "fully pre-registered mirrored result" and "all eligibility gates passed". Under a reanalysis specification and decision rule frozen before any side-stratified number was computed, the headline survives as post-hoc robust: +4.56pp after 50/50 side standardization (CI [2.89, 6.25]), with the valid-only and within-question estimates agreeing in direction. These intervals are question-cluster bootstraps under the observed assignment and do not capture uncertainty over the missing opposite-polarity outcomes. The evidence-content contrast is stable; the packaging contrast is positive under the strict rule but inconclusive under valid-only. The secondary contrasts are re-examined in the report.

Phase 3 launched on September 6. It reuses the 492 blind transcripts across oracle budgets 0, 1, 2, 4, and 8 with two judge endpoints, Qwen3.8-2.4T-A95B and Llama-3.3-70B, for 9,840 judgments: four budget-versus-zero error contrasts under Holm, plus a secondary comparing budget 8 with budget 2. Screening reduced seven candidate judges to two, so the confirmatory claim averages these two fixed endpoints equally, with no capability-slope inference and no generalization to scale.

Delegation disclosure. While I was away, Claude launched the run under my written Phase 3 spending and execution delegation and generated the authorization signature using my key. The record identifies this as delegated approval and does not assert my personal review or signing of that artifact.

Spend. About $208 through July 14 and $181 for Phase 2. Phase 3 pre-main accounting is $119.27 ($95.68 settled, $23.59 uncertain reservations); Together's console shows $94.07 for the same window. The main run had spent about $17 as of September 6, 23:50 UTC. The $1,100 Phase 3 cap includes the pre-main spending; the certified main-run forecast is $908.82.

• Phase 2 report with erratum: https://github.com/jackmaiorino/selvarath-debate/blob/34898fb/reports/2026-08-11-phase2-main-results.md

• Phase 3 protocol: https://github.com/jackmaiorino/selvarath-debate/blob/2707cd3/rejudge/phase3_protocol_v3_r6.json

• Delegated signature record: https://github.com/jackmaiorino/selvarath-debate/blob/2707cd3/rejudge/phase3_main_delegated_signature_provenance_2026-09-06.json

What are your next steps?

Results or a status update in about a week, either way, under the frozen analysis.

Is there anything others could help you with?

Methods scrutiny of the erratum's post-hoc side standardization and of the two-endpoint claim language for Phase 3.

What progress have you made since your last update?

The validation run from my last update is complete. We re-judged the same 318 pilot transcripts under six controlled arms: 11,330 judgments for $182.16.

With the oracle and judging bugs fixed, a few oracle calls still increased judge error by 3.4 percentage points (95% CI [1.3, 5.9]), about half the original 7.2-point estimate. Bug replay indicates that the two oracle-channel bugs contributed roughly half the original effect. Information-free placebo turns caused a smaller 1.6-point degradation, and the pilot's apparent recovery at larger oracle budgets did not reproduce.

Under the frozen gates, the original >=4-point primary gate was indeterminate, not passed. The bug attribution gate technically fired by 0.01 points, while the amendment frozen before the K=3 data passed and licensed only a reduced-scope Phase 2 pilot. The design below is that reduced-scope follow-up, not an unqualified launch of the originally proposed full grid.

One distinction matters: Stage 1 cleanly re-judged the legacy transcripts, but it did not repair a separate debate-generation problem in which each debater had advance knowledge of the opponent's upcoming case. Phase 2 therefore regenerates every debate with blind opening turns.

Follow-up mechanism work also changed our interpretation. Replaying the same query/answer evidence as a neutral table in fresh context removed a large share of the harm. This points to conversational presentation, turn structure, or commitment to earlier questions as part of the mechanism, rather than bad factual content alone.

Held-out calibration selected blind, uncapped, three-round debate. The selected roster has judges Qwen2.5-7B, Gemma-4-31B, Llama-3.3-70B, and GPT-OSS-120B; the debaters are Llama-3.3-70B and hosted Qwen3.7-Plus; the oracle is Llama-3.3-70B. The approved offline design contains 23,200 Phase 2 cells: a 1,060-cell capability preflight followed, if the later gates pass, by 22,140 post-canary main cells. It includes a full cap-protection interaction, an empty-evidence diagnostic, and two full-document gold-context anchors. No calls from this approved 23,200-cell plan have begun.

Previously reported project spend is approximately $208. Across two transfers, I have moved $1,800 of the grant into prepaid Together API credit ($500 earlier and $1,300 now). The Together dashboard showed $1,590.78 in available credit on July 16, implying $209.22 in net credit draw. I will reconcile the $1.22 difference from the rounded spend report against provider usage before another call. Transfers are funding, not experiment spend.

• Stage 1 protocol: frozen protocol

• Stage 1 results: validation report

• Mechanism memo: mechanism and packaging memo

• Calibration report: held-out calibration results

What are your next steps?

Before any call or outcome from the approved Phase 2 plan, owner-approved Amendment A1 waived the planned 106-question human confirmation pass. The reported full/partial/none strata are now explicitly algorithmic reply-pattern classes. A source-bound AI-assisted audit verified all 106 mappings (6/41/59; zero disagreements); its 63 semantic-quality flags are exploratory limitations only. These classes are not human-validated, and the annotations cannot relabel, filter, exclude, or reweight questions.

Before examining any Phase 2 debate outcomes, we will validate the separate query checker against a human-labeled set and freeze/hash the prompts, model settings, exact inventory, retry rules, provider reconciliation, and spending controls.

After this update, we will separately recover or waive 11 failed Gemma calibration cells (proposed recovery cap $2). A separately authorized capability preflight (five models x 106 questions x two mirrored answers; proposed cap $15) will then select the second full-document anchor without examining Phase 2 debate outcomes. Only after that will we separately authorize a small manifested canary on questions excluded from the main analysis. We will review validity and parsing, side bias, provider reliability, artifact completeness, resumability, and ledger-versus-provider cost before separately authorizing the main run. Any failed gate stops the experiment before main spend.

The current provisional empirical planning band is $650–$1,150 in incremental Phase 2 spend, with a $1,200 operational working budget and an immutable $1,500 incremental hard ceiling. We will replace the provisional band with frozen prompt/token profiles and current provider prices before authorizing paid work. The ceiling is a fail-closed boundary, not expected spending. We will post results, failures, artifacts, and actual spend either way.

• Phase 2 design and budget: immutable protocol and cost model

• Resolvability amendment: 106-question AI-assisted audit and owner-approved Amendment A1

• Launch readiness: updated readiness and sign-off
  

Is there anything others could help you with?

Methods scrutiny before the canary, especially the H/P/R decomposition, capability measurement, query-screen validation, clean-versus-placebo comparison, and stopping rules, would be valuable. Pointers to related work on verification interfaces, conversational presentation effects, or deliberation-induced degradation are also welcome.

What progress have you made since your last update?

We re-analyzed the pilot data and quantified the headline effect: a small oracle budget raised the dishonest debater's win rate by +7.2pp (95% CI [4.6, 10.2]).

Before scaling up, we audited the pilot code and found two serious bugs in the oracle channel: every "NOT ADDRESSED" reply was miscoded to "NO", and ~100% of oracle queries were sent garbled. We retracted the mechanism conclusions and corrected the write-ups.

We then rebuilt the harness and launched a pre-registered validation run (in flight now, ~$150-200 of the grant): the same 318 transcripts re-judged under six arms, including a clean harness, a faithful bug replay, and a placebo oracle. Gates were frozen before any clean data existed. Pre-registration: https://github.com/jackmaiorino/selvarath-debate/blob/rerun-new-models/docs/rejudge-protocol.md and corrected report: https://github.com/jackmaiorino/selvarath-debate/blob/rerun-new-models/reports/2026-07-06-preliminary-findings.md

What are your next steps?

The pre-registered gates decide: if the effect survives the clean harness, we proceed to the proposed judge x debater capability grid (most of the grant). If the placebo explains it, we pivot to studying deliberation effects. If it collapses, we publish the artifact result and a decomposition of what each bug contributed. Write-up either way, including negative results.

Is there anything others could help you with?

Methods scrutiny of the pre-registered protocol before results land, and pointers to related work on oracle/verification interfaces or deliberation-length effects in LLM judging.