Manifund foxManifund
Home
Login
About
People
Categories
Newsletter
HomeAboutPeopleCategoriesLoginCreate
🌻
🌻
udbhav bharti

@connectwithub

$0total balance
$0charity balance
$0cash balance

$0 in pending offers

Projects

GharBench - Benchmark of LLM sales agents under Indian law (WhatsApp, Hinglish)pending grant agreement signature

Comments

GharBench - Benchmark of LLM sales agents under Indian law (WhatsApp, Hinglish)
🌻

udbhav bharti

about 9 hours ago

Funny story from the pilot, I had tested the buyer personas to check which model I should use to be the buyer simulator. And guess what? In that pilot run itself, we've got a bug for just $2.27, and that was not just a small bug, but a major one in which one of the buyer models that we were testing gave the hidden instructions that were passed to it in the chat itself, making the whole conversation moot. The automated checks couldn't detect it and were caught later in a blinded human validated test. This is now a fixed gate. I have published all these things and the rest of the numbers on my GitHub repository, where you can clone the repository and do the test runs for yourself and validate whether they are reproducible or not. Lastly, if you have any further questions, I'm always open to discussion as this is my very first research project.