What the survey found
Each point is backed by a profile in the Profiles tab.
Open clones overtook Jev within a month
Every board entrant by the date its HF repo was created; the solid line is the best open score so far. Colored dots are profiled models.
How several questions share one state
The biggest architectural split. Jev documents that the state is read once and each question is isolated. Click a model to see its details.
Size buys score
Decision Index score against parameters (log scale), every board entry with a known size. Hover for details.
Release wave
First upload per model family against total downloads (log scale). 197 families since Jev launched.
Decision Index leaderboard
Colored bars are models profiled here, by multi-question pattern; grey bars are other entrants. Jev is outlined.
Which base model wins
Entries per base model and their best score. Bases with at least 3 entries.
Fine-tune method
Score distribution per training kind. Each dot is one entry; the bar marks the median.
Best score per size class
Top entry in each parameter band. Sub-1B models stay far behind.
Board vs self-reported
Profiled models with a score. Grey bars are self-reported numbers from model cards; they are not comparable with the board.
Architecture matrix
Every profiled model on one row. Sort by any column, filter by group or pattern, click a row for details.
Readout × multi-question pattern
How the hidden state becomes probabilities, against how questions share the state.
Readout heads
What each family puts on top of the backbone.
Options per question
Largest option set each model accepts (log scale). Zeiger is the only one past Jev's 255.
Attention in the backbone
Causal decoders dominate; several labs switch full-attention layers to bidirectional.
Skill against calibration
Decision Index score against expected calibration error (lower is better), board entries with ECE. Top left is the target.
Calibration schemes
How profiled models turn logits into probabilities they stand behind.
Training signal
Where the labels come from. Distilling Jev outputs is forbidden by TypeSafe's agreement §2.3(b).
ECE distribution on the board
Calibration error of all entries with a value, in bins of 0.02. Jev 1.14 sits at 0.080.
Backbones under the clones
Text model type in the config of each HF repo found (repos with a readable config).
Layer mix
Attention layout of those configs. Recurrent DeltaNet layers cannot honour a block-causal mask, which pushed clones to cached prefixes.
LoRA rank
Rank in each adapter_config.json found.
Families by kind
Who builds Jev clones.
Most downloaded families
Total HF downloads across a family's repos, top 15 plus Zeiger. AutoTrust (distilled from Jev) leads by an order of magnitude.
What this means for Zeiger
From COMPARISON.md §10.
- Keep chunked long context and the option set module. No other clone handles pages with more than 255 elements.
- Move to a shared state cache, one row per question. Qwen3-0.6B is pure attention, so a KV fork or kev's block-causal mask works without DeltaNet complications.
- Test order bias cheaply. Drop positions in the set module (ejhshen/OpenJev) or average two option orders (Xor).
- Richer calibration. Per-type temperatures (Strands) or input-dependent ones (Von) next to our option-count buckets; basal's certified thresholds fit the jevpilot action gate.
- From WebJev: jev-ultrafast's wire format and 125-task live benchmark, a separate operation/DONE question, richer element fields, hard negatives.
- CPU tier: Crivo-150M's distillation recipe and Decima's order-free, linear-cost scoring.
- Stay clean: no Jev-distilled data and no benchmark train splits.
What to try first
Each dot is an architecture decision taken from the clones, placed by expected gain for Zeiger against cost. Top left first. Click a dot to jump to its card.
Which weakness each decision attacks
Zeiger's measured problems (laya-browser roadmap), with the summed expected gain of the decisions aimed at each.
Which base models each decision works on
Most decisions carry over to any backbone; the ones that don't decide which base models stay worth trying.
Ranked decisions
Builds on the laya-browser roadmap of 22 Sep (R4, T3, A1 … refer to it). The same list is in ZEIGER_NEXT.md.