These are measurements published by project maintainers and community researchers. Atlas has checked the linked reports and transcribed the aggregate data; we have not rerun model inference.
Inspect the source ↗Laya accuracy benchmarks by task
The project’s application run used Laya 0.2.1 on CPU, 400 cases per task, seed 13, on September 19, 2026. The table is generated from the archived JSON, with each checkpoint shown separately.
| Task | English | Multilingual | Typed decisions | Dataset exposure |
|---|---|---|---|---|
| Email spam | 99.25% | 99.25% | 95.75% | Source in training mix |
| Phishing | 98.00% | 99.25% | 94.00% | Source in training mix |
| Support triage | 50.25% | 52.25% | 50.50% | Source in training mix |
| Toxicity moderation | 53.00% | 52.50% | 53.00% | Source held out |
| Jailbreak detection | 70.75% | 75.50% | 76.25% | Source held out |
| RAG relevance | 62.50% | 65.75% | 62.50% | Source in training mix |
The important contrast: spam classification is near 99%, while toxicity moderation is around 53%. The high spam result comes from a data source in the training mix. Neither number establishes performance on your incoming traffic.
“Source held out” means the project reports that the source was absent from its training mix; it does not establish an independently audited contamination-free test.
Jev accuracy and latency: an independent pilot
AbdelStark’s jev-benchmarks tests Jev 1.13.0 against GLiNER2.5 on 100 examples for each of three conditions. Both models receive the same examples and label descriptions. These are not the 400-case Laya runs above.
| Condition | Jev accuracy | GLiNER2.5 accuracy | Jev p50 |
|---|---|---|---|
| AG News · 4 labels | 91% | 70% | 256 ms |
| Banking77 / BTZSC · 72 labels | 87% | 61% | 246 ms |
| DAIR Emotion · 6 labels | 48% | 44% | 236 ms |
Jev is called from France; GLiNER runs locally on an Apple M4 Max CPU. Latency includes these different deployment paths. On emotion classification, Jev assigned zero probability to the correct label on 16% of examples.
The small pilot supports follow-up testing, not a universal ranking. Its ≤5% error coverage thresholds were selected and evaluated on the same slice, so they should not be copied into production.
Laya vs Jev on 64 Chinese workplace scenarios
Adkid-Zephyr’s contribution preserves model responses for 64 AI-assisted synthetic scenarios, eight scenario families, and four balanced labels. The following scores use the preselected first repeat from September 21, 2026.
| Model / checkpoint | Choice | Four yes/no questions |
|---|---|---|
| Laya multilingual | 20 / 64 · 31.25% | 18 / 64 · 28.13% |
| Jev 1.13.0 | 64 / 64 · 100% | 63 / 64 · 98.44% |
Laya used the multilingual checkpoint at 1c5edc17 on M4 MPS, float32. Jev used the hosted API. These are historical checkpoint results, not scores for today’s main branch. A later Jev Choice repeat scored 63/64.
Use the failures to choose your next experiment
For a support router, prioritize support-triage evidence over spam accuracy. For Chinese workflows, preserve language, label policy, and prompt format in your own test. For performance, compare end-to-end paths with the same payload and concurrency.
Do not combine these tables into “Laya beats Jev” or the reverse. The separate application and pilot studies differ in dataset slices, checkpoint choice, and label counts; only the paired diagnostic above shares its task inputs.
Sources & review notes
Source review: September 27, 2026. Product documentation can change. Atlas has not rerun model inference. Third-party measurements, where included, are attributed to their authors.
- Laya benchmark report — maintainer and contributor measurements
- jev-benchmarks — independent pilot, protocol, and limitations
- Chinese workplace diagnostic — archived paired synthetic-case evaluation