You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
I built beancount-ledger, a multi-turn RL environment for tool-using agents. The basic setup is simple: the agent is given the books of a small company and has to do the job of a bookkeeper. It reads the existing ledger, checks it against a bank statement and supporting registers, works out what is missing, duplicated, or incorrect, fixes the ledger, and submits the result.
One of the main reasons I built the environment this way is that I wanted the score to be unambiguous. There is no LLM judging another LLM. The final accounting state is checked deterministically: the books either reconcile correctly or they do not.
The environment is already live on Prime Intellect's Environments Hub. I have also built a keyed generative population of 1,400 preflighted tasks and published the evidence behind the benchmark rather than only publishing headline scores.
What I want to fund now is the next stage of the project: making the environment usable through the UK AISI Inspect framework and running a much broader evaluation across open-weight models.
The main goal is to turn beancount-ledger from a working environment into a benchmark that other researchers can reproduce, inspect, and run without having to trust my interpretation of the results.
The first step is an Inspect port. UK AISI's Inspect framework is already used by people working on model and safety evaluations, so supporting it would make the environment much easier to run alongside existing evaluation suites.
The second step is a larger model comparison. My initial pilot covered four models and produced 138 valid rollouts, but I had to stop when the free inference quotas I was using ran out. I want to rerun the work properly across roughly 15–20 open-weight models, with multiple rollouts rather than relying on one attempt per model. I will preregister the measurement setup before running the final comparison so that evaluation choices are fixed before I see the results.
The third step is scorer hardening. I already maintain adversarial cases designed to find situations where an agent could receive credit without actually completing the bookkeeping task correctly. As I add the new task family and expose the environment to more models, I want to expand that exploit corpus as well.
Most of the engineering is already complete. The remaining limitation is mostly the cost of running enough inference to make the measurement useful.
I would use the grant almost entirely for inference and compute.
Around $1,000 would go toward API/inference credits for the main evaluation across 15–20 open models, including repeated rollouts across the released task population.
Around $600 would cover compute needed for the Inspect integration, task-generation sweeps, testing, and CI.
The remaining $400 would go toward adversarial runs aimed specifically at finding scorer exploits or edge cases before the final benchmark release.
I am requesting a relatively small amount — $2,000 minimum / $8,000 maximum — because I am not trying to fund salaries or rebuild the project from scratch. The environment, tooling, task generator, scorer, and initial evaluation already exist. I built those independently and have not paid myself for that work.
If additional funding above the minimum is available, I would use it to increase model coverage and rollout count rather than expand the project scope.
I am the only person working on the project.
I built beancount-ledger end to end: the generative world and task system, deterministic accounting scorer, adversarial exploit tests, release tooling, and the initial four-model experiment. That pilot produced 138 valid rollouts, and I designed the evidence chain so that reward claims can be traced back to explicit accounting oracles rather than an opaque model judge.
Source and evidence are public here:
https://github.com/gultekinhasancan79/beancount-ledger
The live environment is here:
https://app.primeintellect.ai/dashboard/environments/cangultekn/beancount-ledger
My professional work has also involved evaluating agent behavior rather than only building software. At Fleet AI, I worked on agentic-task QA and separated more than 100 automated-verifier defects from actual agent failures. I also do RLHF code-preference evaluation at Alignerr across 15 Python libraries.
I am based in Turkey and have a BSc background in Robotics & AI.
The most realistic failure mode is that the project still ships, but the evaluation is smaller than I want.
That already happened once during the pilot: I reached the limit of the free inference resources I was using and had to stop after four models. If funding is insufficient, I could end up with another useful but narrow model comparison instead of the 15–20 model measurement I am aiming for.
The Inspect port is another possible source of delay. If integration work takes substantially longer than expected, I would prioritize publishing the broader measurement and complete the port separately rather than block the entire release on one integration.
There is also a chance that testing many more models reveals a scorer edge case that did not appear in the initial 138 rollouts. I would not consider that a wasted result. A benchmark like this should be attacked before people rely on it. If a model exposes an exploit or ambiguous scoring path, I would fix it, add the case to the regression corpus, and document what happened publicly.
The important point is that the project does not disappear if one of these things happens. The environment is already built and publicly available. The main downside would be a narrower or slower measurement release.
I have not raised any money for this project so far.
Everything currently public — the environment, task population, scorer, pilot experiment, and supporting tooling — was built using my own unpaid time and free-tier infrastructure.
I also submitted a BlueDot Rapid Grant application this week. It is still pending and I have not received funding from it.
I am disclosing both applications because I do not intend to fund the same expense twice. If either application is successful, I will report that and separate the funded line items accordingly.