A Jev-inspired game: 10 statements, true or false. Your probabilities against a small AI's — scored on calibration and points. Runs entirely in your browser. No API key. No payment. No sign-up.
10 statements. Some true, some false. Lock in your probability, see the engine's, then decide: AUTO-ACT, ESCALATE, or SKIP. You're scored twice — on calibration (Brier score, lower wins; the same objective RLCD trains on) and on points.
The engine is a small AI language model running entirely in your browser — nothing leaves your device. For each statement it asks itself: does this read more like something true, or something false? It scores both, squashes the two into one probability, and that's its guess.
It does not look anything up. It's pattern-matching on language — which is exactly why it can be confident and wrong. The points punish that: act on a wrong guess and it's −200.
Points: AUTO-ACT is high risk, high reward (+100/−200). ESCALATE is the safe middle (+20/−10). SKIP is 0. The 33%/70% lines above aren't chosen — they fall out of the math given these stakes. Change the stakes and the lines move.
Calibration: after each round, both you and the engine get a Brier score — the average of (your probability − truth)². Lower is better. It's a proper scoring rule: your best strategy is to report what you actually believe, not to game it. Over 10 rounds, whoever's probabilities match reality better wins calibration — even if they lose on points.
The engine: a small open-weights language model (Xenova/nli-deberta-v3-xsmall) running 100% in your browser via Transformers.js. For each statement it scores how much the text reads "true" vs "false" and normalizes the two into a probability. It doesn't look anything up — it guesses from language patterns, which is exactly why it can be confidently wrong.
The scoring: the Brier score — mean((your probability − truth)²), lower wins — is a proper scoring rule: your best move is to report what you actually believe. It's also the objective behind RLCD, the training method behind the real Jev — the reason Jev's probabilities are meant to mean something.
The thresholds: 33% and 70% aren't chosen — they fall out of the payoff table. Change the stakes, the lines move. That's the real lesson: autonomy thresholds come from economics, not vibes.
What's not real Jev: the real Jev trains its probabilities to be calibrated; these are raw scores from a tiny model — demo-grade. But the duel measures calibration live instead of asserting it, which is the honest version of the claim.