Project summary

Title

HB-Eval: From Measuring Agent Failures to Open Operational Reliability and Assurance Infrastructure

Subtitle

An independently bootstrapped research program built from empirical evidence, open reproducibility, and deployed infrastructure—now advancing toward independently validated reliability assurance for Agentic AI.

Project Summary

We are giving AI agents more autonomy every year. They plan across multiple steps, call external tools, interact with changing environments, and increasingly operate in workflows where a single failure can propagate through an entire trajectory.

Yet most evaluation still asks a surprisingly limited question:

Can the agent complete the task?

I started HB-Eval because I believe that question is not enough.

An agent may complete a task under ideal conditions and still behave unreliably when its plan degrades, a tool fails, constraints conflict, or failures begin cascading through execution.

Capability and operational reliability are not the same property.

I did not begin by building a platform and then searching for a scientific problem to attach to it. I started by measuring the problem.

My first HB-Eval study examined agent behavior through 500 evaluation episodes and found a 42.9 percentage-point gap between nominal success and stressed operational performance. That initial work established the empirical motivation for the project and became the starting point of a much larger research program.

Look at

https://www.preprints.org/manuscript/202512.2186

The next study went substantially further. My primary HB-Eval manuscript reports 14,000 evaluations, conducted across three independent methodologies and 14 model variants. The scale matters because I did not want the argument to depend on a handful of illustrative failures. The results showed that nominal capability and operational reliability cannot simply be treated as interchangeable: cascading faults produced a 21.6 percentage-point reliability penalty, while aggregate operational reliability remained far below what nominal task-success numbers alone would suggest.

The full study is publicly available at:

https://doi.org/10.20944/preprints202606.0186.v1

The manuscript is also currently under review at IEEE Transactions on Artificial Intelligence, manuscript ID TAI-2026-Jun-A-01443.

But measuring the gap was only the beginning.

Over time, the research developed into a connected scientific architecture:

• HB-Eval — Measurement: measuring operational reliability through multiple dimensions rather than reducing agent behavior to a single success rate.

• Adapt-Plan — Control: using evaluation signals to support adaptation when planning behavior begins to deteriorate.

• Look at

• https://www.preprints.org/manuscript/202601.0038

• Evaluation-Driven Memory (EDM) — Persistence: treating long-term memory as a governance problem. EDM is deliberately selective; a trajectory should not become future precedent merely because it happened. Operational experience must first satisfy reliability conditions before being admitted into persistent memory.

• Look at

• https://www.preprints.org/manuscript/202601.0195

• HCI-EDM — Evidence-Grounded Explanation: connecting explanations to evaluated operational evidence through semantic similarity rather than relying entirely on post-hoc generated rationales.

• Look at

• https://www.preprints.org/manuscript/202601.0896

This created a closed research loop:

Measurement → Control → Selective Persistence → Evidence-Grounded Explanation.

I then took the next step: I turned the research into open infrastructure.

The core implementation and reproducibility work are openly available at:

https://github.com/hb-evalSystem/HB-System

The project is also deployed as an operational platform at:

https://hbeval.com

HB-Eval is intentionally not just a web dashboard.

It supports two complementary paths for evaluating agents. Developers and researchers can use the local evaluation path, integrating the SDK into their own infrastructure and observing reliability during execution. The platform also provides a platform-run evaluation path, creating an independent execution environment rather than relying exclusively on self-reported results from the agent operator.

The deployed platform already brings together operational capabilities including live reliability monitoring, controlled fault injection, policy-driven responses, cooperative Safe Halt mechanisms, reliability baselines, evaluation history, CI-oriented reliability gates, audit-oriented evidence, and reproducible evaluation workflows.

It also introduces three visible reliability tiers—Tier 1, Tier 2, and Tier 3—as evidence-based operational qualifications. I do not claim that these tiers are already universal or externally accredited safety certifications. They are not. One of the purposes of this funding is to expose their thresholds and semantics to independent scientific validation.

The same principle applies to the HB-Eval Agent Passport.

I do not believe that an evaluation result should simply disappear inside a private dashboard. The Agent Passport turns operational evidence into a signed, inspectable record of how an agent behaved over time: reliability measurements, policy and Safe Halt events, alerts, and supporting operational history can be presented through a verifiable record.

That distinction matters. A convincing answer is not evidence of reliable behavior.

The project has now entered its next stage: HB-Eval V2.

V2 is not a cosmetic update and not simply a new benchmark. I am building it toward a more mature Operational Reliability and Assurance Architecture for Agentic AI.

The current V2 architecture is increasingly centered around several connected layers of assurance:

• Runtime-Observed Evidence: recording what the evaluation runtime actually observed during execution, rather than relying only on a final model response.

• Controlled Fault Injection: examining how agent behavior changes when execution conditions are deliberately disrupted.

• Multi-Metric Reliability Measurement: preserving distinctions between capability, reliability, recoverability, traceability, constraint compliance, and observed behavior rather than collapsing everything into one score.

• Evidence and Provenance Architecture: tracing exactly which runtime evidence contributed to measurements through typed provenance relationships involving execution steps, fault events, and episodes.

• Integrity-Bound Evidence: using evidence envelopes and cryptographic integrity mechanisms to make tampering detectable under declared assumptions.

• Reliability Baselines: constructing immutable reference states from complete evidence so future comparisons can be grounded in identifiable evaluation conditions rather than arbitrary scores.

• Future Behavioral Constraint Assurance: extending the architecture toward evaluating what an agent actually did relative to what it was authorized and constrained to do—without pretending that unobservable “intent” can be reliably inferred.

This last distinction is particularly important to me.

I am not trying to build another system that claims to detect whether an AI agent has “malicious intent.” Intent is often not directly observable. The more defensible scientific question is: What action occurred, what was authorized, what evidence was observed, what decision was made, and did enforcement actually prevent or allow an effect?

That is the direction HB-Eval V2 is moving toward.

I have bootstrapped this entire journey independently: the original experiments, the research program, the published preprints, the 14,000-evaluation study, the open-source infrastructure, the SDK, and the deployed platform.

Now the project has reached a point where individual bootstrapping is no longer enough.

I am raising $400,000 not to test an initial idea. The empirical foundation already exists. The open implementation already exists. The platform already exists.

The next phase requires resources beyond independent bootstrapping, specifically:

• Independent reproduction of the 14,000-evaluation research program.

• External adversarial testing and broader experiments across frontier models and agent architectures.

• Sustained fault-injection infrastructure capable of supporting larger and more diverse operational experiments.

• Compute and API capacity for large-scale evaluation.

• Additional engineering to harden the V2 evidence, provenance, baseline, and assurance architecture.

• Independent scrutiny of the reliability tiers and Agent Passport model.

• Continued development of the open scientific infrastructure so that researchers can reproduce, challenge, improve, or falsify the framework.

I do not want HB-Eval to become a standard because I declare it to be one.

Standards are not created by self-declaration. They are earned through independent testing, reproduction, criticism, and adoption.

That is why this funding matters.

The project began with a simple scientific question: Can an AI agent finish a task reliably when the real execution conditions begin to break down?

The research produced evidence.

The evidence produced a framework.

The framework became open infrastructure.

Now I want to put that infrastructure under the scrutiny required to determine whether it can become something larger: an open, independently validated foundation for understanding and assuring the operational behavior of Agentic AI.

An agent may finish the task. The harder question is whether it behaved reliably while doing it.

What are this project's goals? How will you achieve them?

My goal is to take HB-Eval from a founder-built research program and working platform into an independently tested, reproducible, and scientifically credible open infrastructure for operational reliability and assurance in Agentic AI.

I want to be clear about where the project stands today, because this funding request is not for an initial idea.

I have already spent the past year building the scientific and technical foundation myself. I began by measuring a problem that I believed was being overlooked: the difference between an AI agent being capable of completing a task and behaving reliably while execution conditions begin to fail.

My first HB-Eval study used 500 evaluation episodes and identified a 42.9 percentage-point gap between nominal task success and stressed operational performance. The original research is publicly available at:

https://www.preprints.org/manuscript/202512.2186
https://doi.org/10.20944/preprints202512.2186.v1

That result did not lead me to write isolated follow-up papers. It led to a connected research path.

I then investigated whether reliability measurement could become a control signal during execution through Adapt-Plan:

https://www.preprints.org/manuscript/202601.0038
https://doi.org/10.20944/preprints202601.0038.v1

After that, I addressed persistence through Evaluation-Driven Memory (EDM), where memory is treated as a selective governance layer rather than a place where every operational experience is automatically retained:

https://www.preprints.org/manuscript/202601.0195
https://doi.org/10.20944/preprints202601.0195.v1

The next step was HCI-EDM, which explored Performance-Grounded Interpretability by connecting explanations to evaluated operational evidence and retained experience rather than relying entirely on convincing post-hoc narratives:

https://www.preprints.org/manuscript/202601.0896
https://doi.org/10.20944/preprints202601.0896.v1

Together, this created a connected research path:

Measurement → Control → Selective Persistence → Evidence-Grounded Explanation.

The empirical foundation then expanded substantially. My larger HB-Eval study reports 14,000 evaluations across three independent methodologies and 14 model variants, examining the difference between nominal capability and operational reliability under disrupted execution conditions. It is publicly available at:

https://www.preprints.org/manuscript/202606.0186
https://doi.org/10.20944/preprints202606.0186.v1

This work is also currently under review at IEEE Transactions on Artificial Intelligence, manuscript ID TAI-2026-Jun-A-01443.

I have also made the research infrastructure available through the open-source HB-Eval repository:

https://github.com/hb-evalSystem/HB-System

And I have deployed the working platform at:

https://hbeval.com

So the question I am asking the funding to help answer is no longer whether I can build the first version of this system.

I already have.

The next question is whether HB-Eval can survive the tests that matter most: independent reproduction, adversarial scrutiny, broader experimentation, and external validation.

I want to achieve five connected goals.

1. Independently reproduce and challenge the empirical foundation

My first priority is external reproduction of the 14,000-evaluation research program.

I do not want HB-Eval to become credible simply because I built it or because my own experiments produced positive results. If the framework is scientifically useful, it should survive independent attempts to reproduce it, challenge its assumptions, test alternative configurations, and identify where it fails.

The funding will allow me to support structured replication work through:

• independent researchers and external contributors;

• replication grants, bounties, or targeted research support;

• reproducibility documentation;

• public experiment configurations and datasets where appropriate;

• independent reporting of successful and unsuccessful reproductions.

If the framework is wrong in important ways, I would rather discover that through open scientific scrutiny than protect it behind proprietary infrastructure.

2. Expand HB-Eval across more real agents, models, and operational conditions

The existing 14 model variants and 14,000 evaluations provide a substantial empirical foundation, but I do not consider them the final boundary of the problem.

I want to expand the research across a broader range of:

• frontier models;

• agent architectures;

• planning systems;

• tool-use environments;

• real agent executions;

• operational fault conditions.

The purpose is not to create another leaderboard ranking models from “best” to “worst.”

HB-Eval asks a different question:

What happens to an agent when execution conditions begin to degrade?

The next phase will therefore expand fault diversity, agent diversity, and execution diversity through controlled fault injection, repeated operational testing, real agent experiments, and broader environments.

I want to understand not only whether an agent completes a task, but whether it can recover, adapt, remain traceable, and preserve reliable behavior when parts of its execution environment fail.

3. Mature HB-Eval V2 into an evidence-grounded operational reliability and assurance architecture

I am currently advancing the project through HB-Eval V2.

V2 is not a cosmetic redesign, and I am deliberately avoiding unnecessary feature expansion. The purpose is to strengthen the scientific relationship between a reliability measurement and the evidence that produced it.

The architecture is increasingly centered around several connected layers of assurance:

• Runtime-Observed Evidence: recording what the evaluation runtime actually observed during execution rather than relying only on a final model response.

• Controlled Fault Injection: deliberately disrupting execution conditions and measuring how agent behavior changes.

• Multi-Metric Reliability Measurement: preserving distinctions between capability, reliability, recoverability, traceability, consistency, and other operational properties rather than collapsing everything into one score.

• Evidence Integrity: making evidence tampering detectable under explicitly declared cryptographic assumptions.

• Typed Provenance: recording which execution steps, fault events, episodes, and evidence contributed to individual measurements.

• Execution and Evaluation Identity: distinguishing what was evaluated, how it was evaluated, and which concrete execution actually occurred.

• Reliability Baselines: constructing identifiable reference states from complete and integrity-verified evidence so future operational states can be meaningfully compared.

The purpose is not to claim that cryptography magically proves reality.

It does not.

My goal is narrower and more defensible: to make the relationship between an operational measurement and its recorded evidence inspectable, traceable, and harder to silently alter.

The next stage of funding will help move this architecture from founder-built implementation toward broader scientific testing and external scrutiny.

4. Strengthen the open infrastructure through both local evaluation and the HB-Eval platform

HB-Eval is intentionally being developed through two connected paths.

The first is local and developer-side evaluation.

The open-source infrastructure at:

https://github.com/hb-evalSystem/HB-System

allows researchers and developers to work with HB-Eval in their own environments. The goal is to support SDK-based evaluation, reproducible experiments, fault injection, and integration into agent development and testing workflows without requiring users to send everything through a closed service.

The second path is the deployed HB-Eval platform:

https://hbeval.com

The platform provides a more structured environment for reliability evaluation, operational workflows, monitoring, evidence records, evaluation history, and assurance-oriented reporting.

I want these two paths to reinforce each other.

A researcher should be able to inspect and reproduce the underlying methodology locally. A developer or organization should also be able to use the platform to manage structured evaluation workflows and examine operational evidence over time.

I do not want to build a closed evaluation service where users submit an agent, receive a score, and are simply expected to trust the result.

The funding will help strengthen:

• reproducible experiment configurations;

• fault-injection environments;

• SDK-based local evaluation;

• platform-based evaluation workflows;

• evidence and audit records;

• CI-oriented reliability gates;

• evaluation histories;

• reliability baselines;

• documentation and reproducibility resources.

The long-term objective is an open scientific ecosystem, not a black-box scoring product.

5. Test the three reliability tiers and the HB-Eval Agent Passport through evidence rather than self-declaration

The HB-Eval platform currently includes three reliability tiers—Tier 1, Tier 2, and Tier 3—as well as the concept of an HB-Eval Agent Passport.

I consider these important parts of the infrastructure, but I do not intend to present them as universal standards simply because I created them.

The reliability tiers are designed as structured operational qualification levels based on evaluation evidence and demonstrated reliability conditions.

The Agent Passport is intended to make an agent's evaluated operational history, reliability evidence, and qualification status more inspectable. Instead of allowing important evaluation information to disappear inside a private dashboard, the Passport provides a way to represent the agent's evaluated operational profile in a more transparent and portable form.

However, these mechanisms still need broader scientific testing.

The funding will allow me to examine questions such as:

• Are the tier thresholds meaningful across different operational environments?

• Do the qualification levels generalize beyond the original experimental conditions?

• Can different evaluators reproduce the same qualification decisions?

• What information should an Agent Passport contain without overstating the level of assurance?

• Which claims can the underlying evidence actually support?

Standards are not created by self-declaration.

If the tiers and Agent Passport are going to become useful infrastructure, they must first survive independent testing and criticism.

How I will achieve these goals

I will pursue the next phase through a staged approach rather than attempting to build every possible feature at once.

First: Independent reproduction and adversarial scrutiny

I will prioritize external reproduction of the existing 14,000-evaluation empirical foundation and actively support attempts to challenge the methodology.

This is the most important next step because the project has reached the point where my own testing is no longer sufficient.

Second: Broader empirical experiments

I will expand experiments across additional models, agents, planning architectures, environments, and controlled fault conditions.

This will include more real agent executions and experiments specifically designed to separate nominal capability from operational reliability and recoverability.

Third: Disciplined maturation of HB-Eval V2

I will continue developing HB-Eval V2 with strict architectural scope, prioritizing:

• runtime-observed evidence;

• evidence provenance;

• integrity verification;

• execution and evaluation identity;

• reliability baselines;

• scientifically defensible comparison methods.

I do not intend to spend this funding on unnecessary cosmetic features or speculative architecture that cannot be empirically demonstrated.

Fourth: Strengthen the open-source and platform ecosystem

I will continue developing both the open-source infrastructure and the deployed platform so that researchers and developers outside my own environment can inspect, reproduce, test, and challenge the work.

The open-source repository and the platform are complementary parts of the same strategy.

Fifth: Validate qualification mechanisms

I will test the three reliability tiers and the Agent Passport as evidence-based mechanisms rather than presenting them prematurely as finished certifications or universal standards.

The goal is to discover what survives broader testing and what needs to change.

The central outcome

The central outcome I am working toward is straightforward:

an open infrastructure that allows researchers and developers to distinguish an agent that can complete a task from an agent that can demonstrate reliable, inspectable, and evidence-grounded behavior while completing it.

I have already bootstrapped the research program, the initial experiments, the sequence of scientific papers, the 14,000-evaluation study, the open-source implementation, and the deployed platform.

The next phase requires resources beyond independent bootstrapping, specifically:

• external reproduction of the 14,000-evaluation study;

• independent adversarial testing and broader experiments across frontier models;

• sustained fault-injection infrastructure;

• compute and API capacity for large-scale experiments;

• additional engineering to harden the evidence and assurance architecture into open scientific infrastructure;

• validation of the reliability tiers and Agent Passport through evidence rather than marketing.

The funding is needed because the next stage cannot be credibly completed by one person testing his own system.

It requires outside researchers, independent reproduction, larger experiments, sustained compute, operational testing infrastructure, and a willingness to discover where HB-Eval is wrong as well as where it works.

That is how I intend to move HB-Eval from a founder-built research and platform infrastructure into something the wider scientific and technical community can actually inspect, reproduce, challenge, and trust on the basis of evidence.

How will this funding be used?

I am requesting $400,000 to support the next 12–18 months of HB-Eval.

I want to be clear about what this money is—and is not—for.

I am not asking for funding to start researching whether operational reliability is a meaningful problem. I have already spent the last year bootstrapping that work myself: conducting the initial research, running the experimental programs, publishing the research path publicly, building the open-source infrastructure at:

https://github.com/hb-evalSystem/HB-System

and developing the deployed HB-Eval platform:

https://hbeval.com

The funding is for the next stage: taking a project that has already moved from empirical research into working infrastructure and subjecting it to the level of independent testing, large-scale experimentation, engineering, and reproducibility work that one independent founder cannot realistically sustain alone.

My planned use of the funding is:

1. Compute, fault-injection infrastructure, and operational experimentation — $110,000

A major part of HB-Eval depends on repeated execution under controlled operational disruption.

That means running agents repeatedly, introducing different fault conditions, recording runtime evidence, comparing behavior across models and configurations, and maintaining reproducible evaluation environments.

I will use this portion of the funding for:

• cloud compute and execution infrastructure;

• scalable isolated testing environments;

• sustained fault-injection experiments;

• storage for evaluation traces and evidence records;

• databases and infrastructure required for reproducibility;

• large-scale repeated agent executions;

• infrastructure needed to test HB-Eval across broader operational conditions.

The existing 14,000-evaluation program established an important empirical foundation. The next phase needs to go beyond that foundation and test whether the findings and methodology remain meaningful across broader agents, models, and execution environments.

2. Frontier model APIs and large-scale evaluation experiments — $90,000

HB-Eval needs access to a diverse range of modern models and agent systems.

I plan to use funding for API access and experimental capacity involving frontier and widely used models where appropriate. This will support evaluation across different model families, agent architectures, planning systems, and tool-use environments.

The purpose is not simply to generate more benchmark scores.

I want to run repeated operational experiments that examine what happens when execution begins to degrade:

• How does the agent respond to tool failures?

• Does it recover?

• Does it adapt its plan?

• Does behavior remain stable across repeated executions?

• What operational evidence supports the final reliability measurement?

Large-scale experimentation across different systems requires sustained API and compute resources, particularly when evaluations involve repeated runs and controlled fault conditions rather than single benchmark prompts.

3. Research, engineering, and independent replication — $150,000

This is the largest part of the budget because the next phase cannot be completed credibly through founder-only development.

I will use this funding to support my full-time research and engineering work on HB-Eval while also bringing external people into the validation process.

This includes:

• continued scientific and engineering development of HB-Eval V2;

• strengthening runtime evidence, provenance, integrity, identity, and baseline infrastructure;

• improving reproducibility tooling and experimental methodology;

• supporting independent reproduction of the 14,000-evaluation research program;

• targeted replication grants, research bounties, or external contributor support;

• adversarial testing of the framework;

• broader empirical validation across agents and operational environments.

This part of the budget is especially important to me.

I do not want the strongest evidence for HB-Eval to remain evidence produced only by me.

The framework needs people who did not build it to test it, reproduce it, and challenge it.

Some of that process may confirm my findings. Some of it may expose weaknesses or assumptions that need to change.

Both outcomes are valuable.

4. Open-source infrastructure, platform development, and reproducibility — $35,000

HB-Eval is being developed as open infrastructure rather than as a closed scoring service.

I will continue investing in both the open-source ecosystem:

https://github.com/hb-evalSystem/HB-System

and the deployed platform:

https://hbeval.com

The funding will support work on:

• SDK-based local evaluation;

• reproducible experiment configurations;

• documentation and developer resources;

• fault-injection tooling;

• evidence and audit workflows;

• evaluation histories;

• reliability baselines;

• CI-oriented reliability gates;

• platform workflows for structured evaluation and monitoring.

I want researchers to be able to inspect and reproduce the methodology locally, while developers and organizations can also use the platform for structured operational evaluation.

The goal is openness without sacrificing practical usability.

5. Validation of reliability tiers and the HB-Eval Agent Passport — $15,000

The HB-Eval platform includes three reliability tiers—Tier 1, Tier 2, and Tier 3—and the HB-Eval Agent Passport.

I do not want to spend funding simply promoting these ideas as certifications.

The purpose of this budget is to test them.

I want to investigate whether:

• tier thresholds remain meaningful across different environments;

• qualification decisions can be reproduced independently;

• the tiers communicate useful operational information;

• the Agent Passport can provide a useful evidence-grounded representation of an agent's evaluated history;

• the underlying evidence supports the level of assurance being communicated.

If these mechanisms are going to become useful infrastructure, they need empirical validation before broader adoption.

Budget summary

Area Allocation Compute, fault-injection infrastructure, and operational experimentation $110,000 Frontier model APIs and large-scale evaluation experiments $90,000 Research, engineering, and independent replication $150,000 Open-source infrastructure, platform development, and reproducibility $35,000 Validation of reliability tiers and the HB-Eval Agent Passport $15,000 Total $400,000

I have deliberately structured this budget around the transition the project now needs to make.

The first phase of HB-Eval was largely possible through independent bootstrapping: identifying the problem, building the methodology, running the experiments, publishing the research, developing the open-source implementation, and deploying the platform.

The next phase is different.

It requires:

• independent researchers;

• sustained experimental infrastructure;

• larger and more diverse agent evaluations;

• compute and API capacity;

• reproducibility work;

• adversarial scrutiny;

• and continued engineering of the evidence and assurance architecture.

• The $150,000 research and engineering line will be tracked separately between project labor, external replication support, and engineering or review expenses. This separation will make it clear how much funding supports the founder’s work and how much goes directly to independent scrutiny.

• If the project receives only the $100,000 minimum, I will prioritize protocol freezing, independent reproduction, reproducibility documentation, and an initial Baseline/Diff/Gate implementation. Broader experiments, larger infrastructure, and wider independent participation will be scaled according to available funding rather than treated as guaranteed deliverables.

I am asking for $400,000 because HB-Eval has reached the point where continuing to test a founder-built system primarily through the founder's own resources is no longer enough.

The next stage is about finding out whether the work survives outside my hands.

That is what this funding will be used for.

Who is on your team? What's your track record on similar projects?

HB-Eval has reached its current stage through a founder-led and independently bootstrapped effort.

I am Abuelgasim Mohamed Ibrahim Adam, the founder and primary researcher and engineer behind HB-Eval. I have personally led the project from its earliest empirical question through research design, experimental execution, software development, open-source implementation, and deployment of the HB-Eval platform.

At this stage, I am the core person responsible for the project.

I do not want to misrepresent that by presenting a large team that does not exist.

What I have built so far was largely built through my own research and engineering work. That is also one of the reasons I am applying for this funding now: HB-Eval has reached a point where founder-only development and founder-only validation are no longer sufficient.

My track record is visible in the progression of the project itself.

I began by investigating a specific question: does nominal agent capability tell us enough about how reliably an agent behaves under operational stress?

The first HB-Eval study examined this question through 500 evaluation episodes and identified a 42.9 percentage-point gap between nominal task success and stressed operational performance:

Rather than stopping with a single evaluation result, I developed the work into a connected research program.

I then built Adapt-Plan, exploring whether reliability measurement could be used as a control signal during execution:

I followed this with Evaluation-Driven Memory (EDM), which investigates selective persistence and treats memory as a governance problem rather than assuming that every past trajectory should become future experience:

I then developed HCI-EDM, exploring Performance-Grounded Interpretability and the relationship between explanations, semantic evidence retrieval, and evaluated operational history:

The research program then expanded into the larger HB-Eval empirical study, involving 14,000 evaluations across three independent methodologies and 14 model variants:

https://www.preprints.org/manuscr

That manuscript is currently under consideration at IEEE Transactions on Artificial Intelligence, manuscript ID TAI-2026-Jun-A-01443.

Alongside the research, I did not leave HB-Eval as a collection of papers.

I built the engineering infrastructure around it.

The open-source implementation is available at:

https://github.com/hb-evalSystem/HB-System

And the deployed HB-Eval platform is available at:

https://hbeval.com

I am now advancing the architecture through HB-Eval V2, focusing on making the connection between operational measurements and their underlying evidence more scientifically defensible. This work includes runtime-observed evidence, controlled fault injection, typed provenance, evidence integrity, execution and evaluation identity, and reliability baseline infrastructure.

My track record, therefore, is not based on claiming experience with a hypothetical future system.

It is the progression of HB-Eval itself:

empirical observation → experimental research → connected research program → large-scale evaluation → open-source implementation → deployed platform → evidence-grounded assurance architecture.

I have demonstrated that I can take an idea beyond a paper.

I can formulate the research question, design the methodology, run experiments, write and publish the research, build the underlying software, maintain an open-source implementation, and deploy a working platform around the research.

However, I also see a clear limit to founder-led work.

I should not be the only person responsible for proving that my own framework works.

That is why the next phase is deliberately designed to expand beyond me.

With funding, I intend to bring external researchers and contributors into the project through independent reproduction, adversarial testing, targeted replication support, broader experiments, and additional research and engineering capacity.

The immediate team today is small because the project was bootstrapped.

The evidence base is not small.

I have already built the research record, conducted the experiments, created the open infrastructure, and deployed the platform. The purpose of this funding is to turn that founder-built foundation into something that can be independently tested and developed by a broader scientific and technical community.

My strongest track record is that HB-Eval already exists. The next challenge is making sure it can survive outside the environment in which I built it.

What are the most likely causes and outcomes if this project fails?

I think it is important to define failure honestly.

For HB-Eval, failure would not necessarily mean that the software stops working or that the research disappears. A more meaningful failure would be that the framework does not survive independent scientific scrutiny, does not generalize beyond its current experimental conditions, or fails to gain enough adoption to become useful shared infrastructure.

There are several realistic ways this could happen.

1. Independent reproduction may challenge the original findings

The most important risk is scientific.

The 14,000-evaluation research program produced significant evidence for a distinction between nominal capability and operational reliability, but independent researchers may find that some results depend more heavily on particular experimental assumptions, models, fault conditions, or evaluation environments than currently understood.

That would be a valuable result.

I would rather discover that HB-Eval has important limitations through open reproduction than protect the framework behind proprietary infrastructure and allow unsupported assumptions to survive.

The likely outcome would be refinement rather than disappearance: metrics might need to change, thresholds might need to be recalibrated, fault models might need to be redesigned, or some claims might need to become narrower.

2. The framework may not generalize sufficiently across future agents and models

Agent architectures are changing rapidly.

Future systems may use different planning mechanisms, memory architectures, tool-use patterns, and execution environments. Some parts of the current methodology could become less informative as agent behavior evolves.

This is why I am treating HB-Eval as an evolving research infrastructure rather than a frozen benchmark.

If the framework fails to generalize, the outcome would be that parts of the methodology need to be updated or replaced. The open architecture, reproducible experiments, and evidence infrastructure would still provide a foundation for that work.

3. The reliability metrics may prove useful but not support universal thresholds

The current platform includes reliability measurements, operational qualification tiers, and the concept of an HB-Eval Agent Passport.

A major risk is that broader experimentation shows there is no single set of thresholds that generalizes across all agents and environments.

I consider this a serious possibility.

Different operational contexts may require different reliability expectations. An agent used for a low-risk internal workflow should not necessarily be evaluated under the same qualification requirements as an agent operating in a safety-critical environment.

If universal thresholds fail, I would narrow the claims and move toward context-specific evaluation rather than pretending that one number can certify every kind of agent.

Standards are not created by declaring them standards.

They have to survive evidence.

4. Open infrastructure may struggle to achieve broad adoption

There is also an ecosystem risk.

Large AI companies may continue building proprietary evaluation systems internally rather than adopting an open framework. Developers may prefer simpler benchmarks, even when those benchmarks measure less about operational behavior.

HB-Eval could therefore remain a useful research infrastructure without becoming widely adopted industry infrastructure.

That would limit the project's impact, but it would not erase its outputs.

The open-source repository:

the deployed platform:

the published research, experimental methodologies, and reproducibility infrastructure would remain available for researchers and developers.

5. The V2 assurance architecture may prove more difficult to validate than expected

HB-Eval V2 introduces a more ambitious engineering direction: connecting operational measurements to runtime-observed evidence, typed provenance, integrity mechanisms, execution identity, and reliability baselines.

There is a risk that some of these components add complexity without providing enough scientific value.

I am deliberately trying to control this risk through strict scope boundaries and empirical validation. I do not want complexity for its own sake.

If particular architectural components fail to demonstrate measurable value, they should be removed, simplified, or deferred.

The project should adapt to evidence rather than forcing the evidence to justify the architecture.

The most likely overall outcome if the project does not succeed

The most likely outcome is partial scientific success but limited broader adoption.

HB-Eval may establish useful findings, infrastructure, and methods without becoming a widely adopted standard. Some metrics or qualification mechanisms may survive independent testing while others are revised or rejected.

That is a risk I accept.

Even under that outcome, the project would leave behind:

• the original 500-episode empirical study;

• the larger 14,000-evaluation research program;

• the connected research path from measurement to control, selective persistence, and evidence-grounded explanation;

• open-source evaluation infrastructure;

• reproducible experimental methodology;

• fault-injection tooling;

• runtime evidence and provenance architecture;

• documentation of both successful and unsuccessful assumptions.

The worst outcome would be spending the funding and learning nothing.

I do not think that is the likely outcome because the project is structured around open experimentation and falsifiability. Independent replication can confirm findings, weaken them, or overturn them. All three outcomes produce useful knowledge.

If HB-Eval succeeds, it may become useful open infrastructure for understanding operational reliability and assurance in Agentic AI. If it fails to achieve that level of adoption, the scientific record, experiments, software, and lessons learned will still remain publicly available for others to build on.

That is why I believe the downside is relatively contained compared with projects that depend entirely on a proprietary product succeeding commercially.

How much money have you raised in the last 12 months, and from where?

$0.

I have not raised external funding, venture capital, grant funding, or institutional research funding during the last 12 months.

HB-Eval has been entirely self-funded and independently bootstrapped.

The work completed so far—including the original 500-evaluation study, the larger 14,000-evaluation research program, the connected research papers, the open-source infrastructure and the deployed platform was developed without external investment.

This funding request would therefore be my first external funding for HB-Eval.

That distinction matters because I am not asking for funding to explore whether I can begin building the project. I have already invested the time and effort required to establish the research foundation, conduct the experiments, publish the research publicly, build the software infrastructure, and deploy the platform.

The purpose of external funding now is different: to move beyond founder-led development and founder-led validation toward independent reproduction, external scientific scrutiny, larger experiments, sustained infrastructure, and broader participation from researchers and engineers outside my own development environment.

I have deliberately taken the project as far as I could independently before seeking significant external funding. The next stage requires resources that cannot be credibly replaced by one person continuing to test his own system.