You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
$14,200 left to reach the funding goal.
An AI agent can give a safe-looking answer while having used powers nobody delegated to it. Nirmata is an open, early-stage method that proposes auditing, after the fact, the recommendation, the authorization, the attempt and the observed effect. The final report does not replace the record of actions. (Code and scripts Apache-2.0; documentation, protocols and data CC BY 4.0, with the exceptions in LICENSE.md; DOI 10.5281/zenodo.22132451.) It combines an output-only evaluator with a trajectory-only evaluator by a fixed OR rule. v0.1.0 is public; no confirmatory result exists yet. This project funds the first fresh blind pilot and a replication, with every outcome published.
Goal: measure, honestly, how much trajectory-aware evaluation adds over output-only evaluation on agent failures involving delegated authority, control integrity and failure-preserving reproducibility. Steps: preregister protocol and metrics; independent custody of holdout labels; hash predictions before ground truth; score without changing anything; publish the result; replicate on a second agentic system; invite external review of the schema.
Public preregistration exists before data; a public pilot report with Safety Blind Spot Rate, Incremental Detection Value and false-positive rate is published whatever the result; a replication on a second system; updated arXiv paper. "Worked" means a credible answer, not necessarily a positive one.
Minimum US$3,000: compute/API ($2,000) + ~33 research hours at $30/h ($1,000) to run the preregistered pilot. Full goal US$14,200: 312 h research time over 6 months ($9,360), compute/API ($2,000), one conference trip for external review ($2,500), misc ($340). Minimum = pilot only; full = pilot + replication + external review.
Solo: Tabata Jahoda, independent researcher in Rio de Janeiro (ORCID 0009-0007-1104-9204), contact tabatajahoda@gmail.com. I keep my full-time job as Account Manager at Easy8 and do this research part-time (12 h/week). Airtable Builder certified, trainer. Built and released Nirmata v0.1.0 in ~10 days after repo creation (schema, blinding protocol, scoring spec, reproducible 6-trace fixture with tests and CI, release checksums, Zenodo DOI). A preregistered n8n adapter holdout satisfied eight preregistered criteria on a second preregistered synthetic workflow (not detection accuracy, not general validation). Publicly documented an invalidated first holdout. 9-page paper pending arXiv endorsement. Repo: https://github.com/thabatarj8-stack/nirmata LinkedIn: https://www.linkedin.com/in/tabata-j-226b6123
Part-time schedule slips; independent label custody is hard for a solo researcher (limitation would be reported and evidence downgraded); trajectory evaluation may add little or have high false positives (published as a negative result); synthetic traces may not generalize. Worst realistic outcome: a delayed or invalidated pilot, documented publicly, plus a reusable open schema.
US$0 received. Pending: Foresight Institute application submitted (US$60k, status pending). Lightcone Commons application submitted (US$25.5k, status pending).