Can Laya run on a CPU?
A September 24, 2026 report ran Laya 0.3.20 on AWS m7a.xlarge: AMD EPYC 9R14, four physical cores, 16 GiB RAM, Linux, no swap. It used fp32, four threads, PyTorch 2.14.0, Transformers 5.17.0, and checkpoint revision 55cf4c4.
| Checkpoint | 1 question | 5 | 10 | 50 |
|---|---|---|---|---|
| english | 580 ms | 3,072 ms | 6,244 ms | 35,969 ms |
| multilingual | 193 ms | 912 ms | 1,842 ms | 11,157 ms |
| typed-decisions | 584 ms | 2,819 ms | 6,031 ms | 35,653 ms |
Each row uses ten timed calls after two warm-ups, in-process rather than over HTTP. Peak RSS was 9.31 GiB for the whole script, with up to five checkpoints loaded. This is not a minimum RAM requirement for a single model.
Does Laya need a GPU?
The benchmark report contains these useful reference points. They are different runs and must not be treated as a normalized hardware leaderboard.
| Published configuration | One-question timing | Measurement boundary |
|---|---|---|
| Tesla T4 · multilingual | 32.8 ms p50 | Project Colab run; ordinary path |
| RTX 4070 Ti SUPER · multilingual | 14.1 ms stock / 2.8 ms fast | 72-token input; agent.predict end to end, incl. tokenization; TileLang fast path |
| NVIDIA GB10 · typed-decisions | 100.2 ms p50 / 169.3 ms p95 | laya 0.3.5, HTTP wrapper, shared GPU; 40 warmed calls |
The RTX fast path requires the optional acceleration stack and has compilation overhead on first use of a new bucket. The GB10 result includes wrapper overhead and a shared workload. A more expensive GPU does not automatically produce a faster single request.
How much RAM and VRAM does Laya need?
The README lists 421M parameters for the English and typed-decisions checkpoints, and 322M for multilingual. Multiplying parameters by bytes per weight gives only a lower-bound planning estimate:
| Checkpoint size | FP32 weights only | 16-bit weights only |
|---|---|---|
| 421M parameters | ≈ 1.68 GB | ≈ 0.84 GB |
| 322M parameters | ≈ 1.29 GB | ≈ 0.64 GB |
These are decimal GB estimates, not measured RAM or VRAM requirements. Add activations, tokenizer/runtime overhead, downloaded files, concurrent requests, and any additional loaded checkpoints. The reviewed report does not establish a universal minimum VRAM figure.
Python 3.10+ is documented. Select a PyTorch build for your actual CPU/GPU platform; installing a package alone does not prove that CUDA, MPS, or XPU is being used.
What configuration should you start with?
For a small local experiment: use an existing machine and one checkpoint, record peak memory and warm latency, then increase request size. A 16 GiB machine is a reasonable experiment target based on the published CPU setup, not a guaranteed minimum or a throughput promise.
For concurrent production traffic: reproduce your queue length, input length, and latency target before renting capacity. Price the utilization you can actually sustain, not the best short-input demo.
Sources & review notes
Source review: September 27, 2026. Product documentation can change. Atlas has not rerun model inference. Third-party measurements, where included, are attributed to their authors.
- Laya benchmarks — hardware-specific conditions and measurements
- Laya README — checkpoint sizes and installation requirements