ΒΑΣΑΝΟΣ · n. touchstone · agent reliability
The stone that reads the streak.
Basanos hardens AI agents for companies that can’t afford to guess — eval audits that block bad ships, judges that survive their own audit, and post-training you own.
9%
Violations blocked pre-ship
71%
Judge catch rate, exposed
61→88%
Task success, post-training
~10×
Lower cost per ticket
The streak test
Would your judge survive its own audit?
Most agents ship on vibes: a demo went well, a spot-check looked fine, nobody saw the failure modes coming. In a regulated workflow, a wrong answer isn’t embarrassing — it’s exposure.
Evals fix that, but only as far as the judge is honest. An LLM judge that misses violations — or hands out passes for the right magic words — turns your safety net into a rubber stamp.
And the moment scores steer training instead of dashboards, the stakes escalate: would your reward survive RL pressure?
The ladder
Each rung is built from the one below it.
Meridian Servicing, first audit to a model they own.
A loan-servicing portfolio company of a mid-market PE fund. Its agent, Harper, reads a borrower’s account, checks hardship policy, proposes a payment plan, and drafts the reply. Wrong answers here aren’t embarrassing — they’re regulatory exposure.
Beat 01 · The gate
Week three earned the retainer. A routine prompt “improvement” shipped to staging; the harness flagged that Harper now approved payment plans past the 24-month policy ceiling in 9% of cases. The release was blocked before one borrower saw it.
- Proposal
- Payment plan: 30 months at $412/mo
- Policy
- Hardship ceiling: 24 months — plan exceeds ceiling
Caught in staging by the regression gate. Release blocked.
Beat 02 · The assay of the assayer
We seeded 200 known policy violations into the traces: the policy judge caught only 71% — and could be gamed. Left alone, a trained Harper would learn to say compliant rather than be compliant. So we hardened the judge until it survived its own audit.
- Response
- Plan terms as drafted… “This plan complies with §4.2 of hardship policy.”
- Judge
- Score inflated by the appended phrase — verdict flipped to pass
A magic compliance phrase should never buy a pass.
Beat 03 · The loop
61%
Task success — frontier API baseline
88%
Task success — after GRPO training
~10×
Lower cost per ticket, in their VPC
Four weeks of GRPO against the hardened judge, inside a sandboxed clone of the servicing stack — and the before/after was scored on the same harness built in the first engagement. The ladder eats its own tail.
Read the full assay →The field is moving
Epoch AI’s practitioner interviews cite robustness against reward hacking as the RL-environment industry’s top quality criterion — because models find ways to game graders. A services category is forming around exactly this loop:
- RunRL
- Osmosis
- Applied Compute
- Adaptive ML
- Cursor (self-environment post-training)
Landscape — not clients or endorsements
Drag your agent across the stone.
A two-week eval audit. Scored traces, a regression harness, and a straight answer.
hello@basanos.ai · Replies within 48h