QuantID Measurement Lab
📐
Measuring AI, science and risk with reproducible methods
QuantID PRO
quantid
AI & ML interests
None yet
Recent Activity
updated a Space 44 minutes ago
quantid/measurement-lab liked a Space about 4 hours ago
FINAL-Bench/ONGRID upvoted an article about 16 hours ago
Leading the System One Mosaic Benchmark: What Darwin-27B-ZTC-v2's #1 MeansOrganizations
None yet
updated a
Space 44 minutes ago
liked a
Space about 4 hours ago
upvoted an article about 16 hours ago
Article
Leading the System One Mosaic Benchmark: What Darwin-27B-ZTC-v2's #1 Means
FINAL-Bench
• • 6 reacted to SeaWolf-AI's post with 🔥 about 20 hours ago
Post
1259
🏆 Darwin-27B-ZTC-v2 just took #1 on the System One Mosaic Benchmark (S1MB).
S1MB compares 102 models across 137 specialized benchmarks, in three task types: Noul (assess a condition), Choice (select an option), Score (rate on a scale). Ranking is by overall Borda score.
📊 Top of the board
🥇 Darwin ZTC v2 (FINAL-Bench) 89.58
🥈 OpenJev-27B 87.50
🥉 AutoJev-27B 87.07
4️⃣ Eikos 27B 85.43
5️⃣ Jev 1.13 85.05
🔎 Ranks 2 to 5 are all the JEV family (TypeSafe AI's System One model, from ex-OpenAI researchers). S1MB exists to compare these System One judges, so leading it is the headline.
⚙️ Why a zero-token judge wins here
🔹 It does not generate. It reads the input and typed questions and returns a calibrated distribution in a single forward pass.
🔹 Zero generated tokens, no decoding loop, so latency and cost stay low.
🔹 Holds up out of distribution too: General Noul 96.00, General Choice 99.34.
It is also #1 on the typed-decisions leaderboard (0.743, zero-shot). Same message from both: a deterministic, calibrated judge at one forward pass per call.
🔗 Model: FINAL-Bench/Darwin-27B-ZTC-v2
🔗 Leaderboard: hotchpotch/S1MB-leaderboard
Standings move as new models are added. Numbers reflect the board at the time of writing. 🙌
S1MB compares 102 models across 137 specialized benchmarks, in three task types: Noul (assess a condition), Choice (select an option), Score (rate on a scale). Ranking is by overall Borda score.
📊 Top of the board
🥇 Darwin ZTC v2 (FINAL-Bench) 89.58
🥈 OpenJev-27B 87.50
🥉 AutoJev-27B 87.07
4️⃣ Eikos 27B 85.43
5️⃣ Jev 1.13 85.05
🔎 Ranks 2 to 5 are all the JEV family (TypeSafe AI's System One model, from ex-OpenAI researchers). S1MB exists to compare these System One judges, so leading it is the headline.
⚙️ Why a zero-token judge wins here
🔹 It does not generate. It reads the input and typed questions and returns a calibrated distribution in a single forward pass.
🔹 Zero generated tokens, no decoding loop, so latency and cost stay low.
🔹 Holds up out of distribution too: General Noul 96.00, General Choice 99.34.
It is also #1 on the typed-decisions leaderboard (0.743, zero-shot). Same message from both: a deterministic, calibrated judge at one forward pass per call.
🔗 Model: FINAL-Bench/Darwin-27B-ZTC-v2
🔗 Leaderboard: hotchpotch/S1MB-leaderboard
Standings move as new models are added. Numbers reflect the board at the time of writing. 🙌
liked a
model about 23 hours ago
liked a
Space about 23 hours ago
liked 9
datasets 1 day ago
Delores-Lin/MDPBench
Benchmark • Updated • 1.62k • 48
joelniklaus/LEXam-hard
Benchmark • Updated • 518 • 917 • 24
LiquidAI/ifstruct-v1.0
Benchmark • Updated • 2k • 566 • 103
MathArena/aime_2026
Benchmark • Updated • 30 • 31.6k • 86
MathArena/hmmt_feb_2026
Benchmark • Updated • 33 • 18.4k • 31
LEXam-Benchmark/LEXam
Benchmark • Updated • 7.54k • 1.69k • 71
llamaindex/ExtractBench
Benchmark • Updated • 370 • 15.7k • 58
MMMU/MMMU_Pro
Benchmark • Updated • 5.19k • 19k • 91
LocalLLaMA/typed-decisions
Benchmark • Updated • 3.2k • 34.5k • 156
upvoted a collection 1 day ago
liked a
model 2 days ago
updated a
Space 2 days ago
liked 2
models 4 days ago