Independent benchmarks with mandatory null controls for living-neural computing claims, plus replication audits of published results. All open-source.
Organoid intelligence — living brain tissue used as a computer — has no shared tests. When a lab claims its dish-grown neurons learned something, nobody can independently check the work. I'm building the answer key: an open benchmark suite with mandatory "trick questions" (null controls) that any claim must pass, plus independent replication audits of three published claims, graded in plain language.
1. Build benchmark suite v1 (months 1–4): open tests with null controls — session-identity controls, label-shuffle nulls, rate-only baselines, chronological splits. Everything public: tests, code, data. 2. Run three independent replication audits (months 5–10): re-run three major published organoid-intelligence claims through the battery and publish grades — pass, fail, or unclear. 3. Publish and hand off (months 11–12): final report, open release, and outreach so the tests become the field's standard homework. Success looks like: the suite used by at least one outside lab, three audits published, and null controls starting to show up in new papers.
$75,000 for 12 months, solo: ~$54k researcher stipend, ~$8.3k self-employment tax, $3k cloud compute and storage, $1.2k software and tools, $2k travel to one workshop, ~$6.5k contingency buffer. The $40k minimum funds a 6-month pilot: the benchmark suite only.
Solo researcher. I built a brain-signal classifier on real organoid recordings that scored 100% — then my own null controls caught it recognizing recording sessions instead of real learning, and I published the failure instead of the headline, with a full open release package (data fingerprints, run tables, code, verification checklist). That's the exact skill this project needs: building the test, then letting the test grade you honestly.
The benchmarks get built but no outside lab adopts them — mitigated by designing the suite around the field's actual published claims so it's immediately useful. Or the audits all come back "unclear" — still a publishable, useful result about the state of the evidence.
$0 — fully self-funded so far. I have grant applications pending with six other funders; this listing is not exclusive.
Comments
Reve Ontiveros
13 days agoFinal week — closing November 10
One week left on this listing, so a short, plain update.
Since the October 23 update, the audit work has done the unglamorous thing it is supposed to do: correct its own record. The Audit #4 (Shao et al.) file-matching re-check is complete. One excluded pair turned out to be a probable filename typo in the deposit; I re-ran the pipeline unchanged with that pair included, the headline numbers moved slightly (paired p 0.3231 → 0.2937), and the verdict did not change — still no significant day-1 to day-3 improvement. The local audit and its README now carry the dated re-check. That is the standard working as designed: when the record changes, the numbers change in public, even when the conclusion doesn't.
The live wash-in study is still preregistered and public (DOI 10.5281/zenodo.23377488), and the lab quotes that will set its budget are still being scoped with the contract labs the earlier updates named.
The honest part, same as last time: if the $40,000 minimum is reached by November 10, the first funded audits and the wash-in study move from scoping to scheduling. If it isn't, the audits continue at the pace self-funding allows — and every result, funded or not, stays public either way.
— Reve
Reve Ontiveros
25 days agoOctober 23 update — three weeks to the deadline
The audit series kept moving this month, and it produced the two results this project exists for — one in each direction.
A claim that survived, and a paper that will be corrected. The prediction registry now has its first scored entry: Audit #8, of a closed-loop cart-pole organoid deposit, came out PARTIAL — within-run learning survives honest stratification, retention does not — and my pre-registered prediction scored a Brier of 0.14. Separately, the authors of Shao et al. 2025 have acknowledged that their classifier was an SVM, not logistic regression as printed, and will ask the journal for a formal correction. That is what an independent referee is for: not just verdicts, but a public record that gets fixed.
The standard is at v0.5, and the next experiment is locked before it runs: the live drug wash-in study is preregistered and published on Zenodo (DOI 10.5281/zenodo.23377488), with reversal as the load-bearing decision criterion. On the lab side, Metrion Biosciences declined the work after a call on October 16 and referred me onward; Axol Bioscience has quoted the study, and I am waiting on the itemized breakdown that will set the Experiment.com budget.
The honest part: this project is still self-funded, and two grant applications were declined this week (EA Funds and Project Tailwind). The Manifund deadline is November 10. If the $40,000 minimum is reached, the first funded audits and the wash-in study move from scoping to scheduling. If it isn't, the audits continue at the pace self-funding allows — and every result, funded or not, stays public either way.
— Reve
Reve Ontiveros
about 1 month agoSix audits in — and the battery just met a headline it couldn't kill.
Audit #3 is done: FinalSpark's live-culture datasets (FS369, FS437). Stimulation reliably evokes short-latency spiking that survives every null applied — but cross-day transfer fails on both datasets, and session identity decodes from background activity alone. The evoked response is real biology; the decoder's memory of it is a session fingerprint. Full release on the repo (report, 15 scripts, all results, 3 plots).
Audit #6 is done too — and it's the first survival in the battery. Liu & Buonomano 2025 (Nature Communications): ex vivo cortical slices trained 24 hours supposedly learn to predict temporal patterns. The paper itself admits the investigators weren't blinded and the batches span a year — a textbook batch-identity trap. It didn't bite: the effect holds within every batch, survives a 0/2,000 stratified permutation null, and transfers across batches at 0.865 leave-one-batch-out. A skepticism machine's failures mean nothing; this battery reports survival when the nulls say survival. That's what makes the other verdicts credible. Citations: GitHub audits/audit-06-liu2025, Zenodo DOI 10.5281/zenodo.23252824.
Eight datasets, six labs, one pattern: within-session effects hold, cross-session transfer fails — except once. The Open Evaluation Standard is now v0.4.
The $75,000 funds 12 months of this: independent audits, the maintained standard, and the benchmark suite. The minimum $40,000 covers six months. Deadline is November 10.
Reve Ontiveros
about 2 months agoTwo shipments this week.
Audit #2 is done. I independently re-analyzed the DishBrain Pong data (Kagan et al. 2022, Neuron) — the famous "neurons learn to play Pong" result. Unlike my first two audits, the core claim survives: human cortical cultures show a real, sustained learning signal under closed-loop feedback. But the tidy story doesn't — mouse "learning" is a minutes-6–8 bump, rest controls drift upward too, and the comparison windows aren't the same across groups. Full write-up lands in the repo.
New video: "AI Is Drinking Our Rivers. The Answer Runs on 20 Watts." — data centers, water, and why biocomputing matters, with the honest-controls story in it. https://youtu.be/dMpThAYcDY8
The standard is public. Open Evaluation Standard v0.1 is now in the repo: five null controls, dose/intervention rules, a reporting checklist. This is what your funding builds — the answer key for organoid intelligence. https://github.com/ontiverosreve/organoid-evaluation-standards