You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
Chain-of-thought oversight assumes the model's written reasoning actually drives its answer. The closest published work (Lanham et al. 2023 on CoT faithfulness interventions; Turpin et al. 2023 on unfaithful explanations) reports faithfulness as point estimates, and to my knowledge neither tests whether the confounding that the shared-network setup structurally guarantees could be driving the number on its own. That gap is what the sensitivity parameter rho is for.
I built a Bayesian causal-mediation auditor for CoT faithfulness. The CoT is treated as a mediator between prompt and answer in a Pearl-style structural causal model; the auditor estimates direct and indirect effects with hierarchical Bayesian pooling; and because one network generates both the reasoning and the answer (so the standard no-confounding assumption is structurally violated), every headline number ships with a sensitivity parameter rho and its breakdown point rho* - the confounding strength at which the conclusion would flip.
Phase 1 was funded by a $650 BlueDot Impact Rapid Grant; the deliverables below are public. The estimator recovers known ground-truth effects on a synthetic testbed (all three 95% credible intervals contain the truth) - a pre-registered go/no-go gate, cleared before any real-model spend. The rho* frontier separates a genuinely causal case (rho* = 0.69) from a decorative baseline (rho* = 0.05). In real planted-hint audits of an open model (Llama-3.1-8B, up to n = 103 per run, free-tier inference), 36.9% of runs followed the planted hint, mostly without acknowledging it in the CoT - and the auditor flagged them (verdicts still heuristic until the golden set below validates them). A real caught transcript is public. MIT-licensed repo with a 124-test CI suite: https://github.com/thylinao1/bayes-cot-faithfulness - interactive demo and evidence: https://thylinao1.github.io/bayes-cot-faithfulness/site/
Phase 2 makes the auditor trustworthy at scale. $800 (minimum) buys the human-labeled golden set that turns heuristic verdicts into validated error rates. $2,500 adds adequately powered GPU sweeps on open models. $5,000 (goal) adds a frontier cross-check on Claude- and GPT-class models.
Validated auditor. Complete the human-labeled golden set: at least 50 transcripts independently labeled by two raters, Cohen's kappa reported. The labeling guide and blinded-sheet generator already exist in the repo; the labeling itself has not been done. Until it is, the auditor's verdicts are heuristics; after it, the auditor reports validated error rates.
Adequately powered open-model runs. Replace the free-tier n = 11-103 pilots with pre-registered, power-calculated sweeps on Llama-3-8B and Gemma-2-9B, including logit-level counterfactual forcing (designed and specified, not yet run - all real-model results so far are text-level interventions on free-tier inference).
Frontier cross-check. Text-level intervention sweeps on Claude- and GPT-class models via API, to test whether the open-model findings transfer across labs. No frontier results exist yet; this buys the first ones.
Public, uncertainty-quantified results. Every model x task cell released with its faithfulness posterior, credible interval, and rho* robustness curve, plus code and transcripts - the seed of a public benchmark. The writeup targets a NeurIPS or AAAI safety workshop.
How: the text-level pipeline already exists end-to-end and passed its pre-registered synthetic go/no-go gate before any real-model spend. Phase 2 is execution against two frozen pre-registrations with explicit pass/fail criteria, not new methodology.
Each tier is independently useful; the tiers nest.
Tier 1 - $800 (minimum funding): validated auditor.
$600: second-rater compensation for golden-set labeling (>=50 transcripts, two independent raters, Cohen's kappa reported)
$200: storage and labeling-infrastructure buffer
Tier 2 - $2,500: adds powered open-model sweeps.
$1,400: cloud GPU compute (roughly 400 A100-40GB hours at ~$3.5/hour RunPod/Lambda spot rates) for logit-level counterfactual forcing and pre-registered powered sweeps on Llama-3-8B and Gemma-2-9B
$300: cloud object storage for activation traces plus re-run buffer
Tier 3 - $5,000 (funding goal): adds the frontier cross-check.
$1,600: Anthropic API credits (Claude sweep, text-level interventions)
$700: OpenAI API credits (GPT-class cross-lab check)
$200: contingency
Total at goal: $5,000.
With minimum funding only, I deliver the golden set and validated error rates, and keep running open-model audits at free-tier scale. With full funding, the open-model sweeps run adequately powered, and the same frozen protocol gets a text-level cross-check on Claude- and GPT-class models. Compute, API credits, and rater time are the entire bottleneck - the method, the text-level pipeline, the pre-registrations, and the guardrails are already built and public; logit-level forcing is designed and specified, awaiting the compute.
I'm an MSc student in NUS School of Computing (Business Analytics), specializing in Statistics, and a winner of the Jane Street Quantitative Reasoning Competition (3x times) and the National Mathematics Olympiad (top 0.015% nationally).
Honest framing - I'm not a computer science expert in mechanistic interpretability, nor do I have famous published research papers on AI safety and causal inference. However, what I am is a passionate applied researcher who spends 6 to 7 hours a day working through statistics, causal inference, reading papers, and iterating through many different ideas that I find genuinely interesting. During my bachelor's, I extensively studied courses from top universities across the world like CS229 machine learning, Stanford CS230 deep learning, Stanford EE178 probabilistic system analysis, completed the entire Imperial College mathematics for ML specialization, and MIT introduction to probability, as well as the IBM applied data science specialization, and now have just finished working through the PhD-level course taught by McElreath and his book Statistical Rethinking, whose ideas I'm actively implementing. One thing that gives me an advantage is a really strong mathematical background going all the way back to when I won the Russian National Mathematics Olympiad in high school, and more recently became one of the winners of the Jane Street quantitative competition puzzle. To showcase my skills and recent work please take a look at some of the projects I have attached below:
Olist Hierarchical Bayesian A/B Testing (PyMC 5, hierarchical difference-in-differences, PSIS-LOO, prior-sensitivity sweeps) and a wait-list-controlled causal-inference capstone that the client firm adopted as its standard for program evaluation (top grade in cohort).
Portfolio: https://thylinao1.github.io/index.html
Track record on exactly this project: BlueDot Impact funded Phase 1 ($650; grant window ends 30 August 2026). Delivered and public so far: the pre-registered synthetic validation gate (cleared - all three 95% credible intervals contain ground truth), the rho sensitivity sweep with its rho* breakdown frontier, real planted-hint audit runs up to n = 103 with a caught Llama-3.1-8B transcript, and a 124-test CI suite on Python 3.10-3.12. The full delivered list (hierarchical Bayesian pooling, positive controls, guardrail audit, two frozen pre-registrations) is in the repo README.
Most likely failure: the golden set shows the auditor's verdicts disagree with human judgment (low kappa). That is a useful result, not a wasted grant - the pre-registration requires two independent raters and a reported kappa before any rate is published as a measurement, and I commit to publishing the error rates either way; the auditor's thresholds then get recalibrated against the human labels rather than quietly tuned.
Second: the powered sweeps find effects too small to detect at feasible sample sizes. The power/MDE machinery is already built and the pre-registrations have explicit pass/fail criteria, so a null gets reported as a null. A measured noise floor for CoT-faithfulness claims would itself be worth publishing - most current claims in this literature have no such floor.
Third: logit-level counterfactual forcing (injecting the counterfactual mediator into the KV-cache of a frozen model) hits engineering problems. It is designed but has never been run. Fallback: the text-level intervention pipeline that already works at n = 103.
Fourth: time. I start an MSc this year. Mitigation: every Phase-1 milestone so far has landed on schedule within the ongoing grant window on 6-7 hours a day, and Phase 2 is execution against existing pre-registrations, not open-ended research.
Worst case, the outputs are still public goods: labeled transcripts, honestly reported negative results, and an MIT-licensed pipeline anyone can pick up.
$650 from a BlueDot Impact Rapid Grant, funding Phase 1 of this project (grant window ends 30 August 2026). No other funding. Everything delivered so far is public in the repo and on the project site.