Project summary

There are problems in frontier AI Safety benchmarks for VLMs as they can often exploit vision shortcuts to win on static benchmarks but failing to remain robust to small changes. Exhaustive fixed-length benchmarks can also cost large amounts by wasting API Calls.

I already developed and open-sourced a project called VLM - TRACE. In the coming 6 months, I am planning to extend this framework to include extra features, like combining it with anytime valid sequential hypothesis engine, mentioned by recently peer reviewed paper for IEEE ICICAIEECE 2027. Planning to install as a python installable library and also as a plug-in for softwares like inspect_ai. By adding these technologies together, it is possible to know safety degradation in VLMs and shortcut exploitation.

What are this project's goals? How will you achieve them?

Make a place where records are stored for e-audits settled my IEEE paper and allure solve plans. Test VLM-Trace on four talk models, holding Hindi, Tamil, Malayalam, thus, utilizing forceful weighing. Share the solver and take form for Inspect AI.Then, check main AI models misusing their APIs, do few fast tests, and issue the early lie of the verdicts.

How will you know it worked?

Results showing 40% fewer API samples vs. fixed-length runs at alpha <= 0.05.

MIT-licensed e-audit + VLM-TRACE repository, passing CI pipeline, and PR.

Push the Public .eval logs and tables on Hugging Face Hub.

Send the Preprint with results on arXiv / NeurIPS 2027

How will this funding be used?

Researcher stipend: $10,200 ($1,700/mo x 6 mos, Trivandrum, India; includes retroactive VLM-TRACE credit).

Frontier VLM/LLM APIs & harness tooling: $3,300 ($550/mo x 6 mos, invoiced).

Contingent IIT Bombay mentorship fee: $2,000 (drawn only with official invoice; otherwise unspent or moved to APIs).

Red-team checks, hosting, workshop travel: $1,900.

RunPod GPU compute: $1,800 ($300/mo x 6 mos, invoiced).

Wire/FX fees & buffer: $800.

Who is on your team? What's your track record on similar projects?

Johan Shiju Varughese — solo PI, final-year B.Tech CS (AI), MBCET, Trivandrum, India.

Built VLM-TRACE; merged PR #3 (BF16 fix) in UBC TEA Lab's vlm-probe.

First author, IEEE ICICAIEECE-2027 (Innovation E-Detector); Zenodo null-result stress test.

Chair, IET MBCET Student Chapter; panel speaker, IET Festival of Volunteering (London, Sept 2026).

Mentorship: IIT Bombay outreach unconfirmed/contingent; applying to SPAR Spring 2027 & AI Safety India.

What are the most likely causes and outcomes if this project fails?

Heavy-tailed VLM noise drops sample savings < 20% -> publish negative results, shift Months 4–6 to shortcut benchmark.

IIT Bombay mentorship falls through -> solo execution unaffected, $2,000 stays unspent.

Low adoption -> native inspect_ai CLI plug-in.

How much money have you raised in the last 12 months, and from where?

$0 USD.