Two models, two jobs. Ternary Bonsai 2 writes. Laya only labels. The card that wins is not the same for both.
Rates below were the list price on the day we measured, and they still match the marketplace on 2026-09-22: A6000 $0.57/hr, L40S $0.97/hr, RTX PRO 6000 Blackwell $2.19/hr. L40S moved from $0.88 to $0.97 on 2026-09-08. Confirm the live rate before you launch. These two models are separate measurements. Different weights, different engines, different units.
What to launch
| You are doing this | Card we measured | Why |
|---|---|---|
| Bonsai chat, agents, short replies | A6000, $0.57/hr | Best output tokens per dollar (117.1). 66.7 tok/s. |
| Bonsai with a long prompt, then a short answer | L40S, $0.97/hr | Prompt ingest (pp512) is a bit more than 2× the A6000. Decode only moves from 66.7 to 75.1 tok/s. |
| Bonsai while a person waits on the tokens | Blackwell, $2.19/hr | 124.8 tok/s, about 1.8× the A6000. Worst tokens per dollar (57.0). |
| Laya on an English ticket queue | L40S, $0.97/hr | 14.3 ms median, 70 classifications a second, 72.2 decisions per dollar. |
| Laya when those extra milliseconds are the product | Blackwell, $2.19/hr | 10.2 ms, 98 classifications a second. Decisions per dollar fall to 44.7. |
The A6000 is the wrong default for Laya if someone is waiting on the label (45.0 ms, 22.2 decisions/s, 39.0 per dollar). It is the right default for Bonsai decode.
Cheaper A6000 listings are in stock and were not launched: spot at $0.50/hr, and low-RAM at $0.55/hr. The $0.57 A6000 is the least expensive card we actually ran, not the least expensive card on the site. A30 ($0.35) and A5000 ($0.44) had no capacity on the capture day and still have none as of 2026-09-22.
Use Bonsai when the model has to write
Ternary Bonsai 2 Ternary-Bonsai-2-27B-PQ2_0.gguf is an Apache-2.0 Qwen3.8 27B with ternary hybrid attention. The file is about 7.21 GB. Full precision is about 54 GB, which does not fit a 48 GB card. The publisher points it at chat, tool use, code, and long documents, with a context window up to 262K on their runtime.
Load it for a private agent or a coding session on one rented GPU. Use the PrismML llama.cpp fork. Stock llama.cpp rejects PQ2_0. We did not run the smaller PTQ1_0 pack or the F16 pack.
What we timed is llama-bench, five repeats. The headline is output token throughput at tg128, not answer quality, and not a 262K context. Measured VRAM was about 7.0–7.8 GiB, so a 48 GB card is not full. You are paying for speed.
| Card | $/hr | Output tok/s (tg128) | tok/s per $ | Prompt tok/s (pp512) |
|---|---|---|---|---|
| A6000 | 0.57 | 66.7 | 117.1 | 1363 |
| L40S | 0.97 | 75.1 | 77.4 | 2922 |
| Blackwell | 2.19 | 124.8 | 57.0 | 4029 |
L40S prompt ingest at 512 tokens was 2922 tok/s versus 1363 on the A6000. Blackwell prompt ingest was 4029 tok/s, just under 3× the A6000. Decode on the L40S is only about 12% above the A6000 (75.1 versus 66.7). Rent the L40S for the prompt. Rent the Blackwell when wall-clock on the reply matters more than the hourly rate.
Screenshots and the raw capture: gpu-benchmark pull 37.
Use Laya when you need a label
Laya does not draft the reply. You pass a state (an email, a ticket, or JSON) and typed questions. One forward returns a department choice, a score, or a probability. There is no paragraph to parse.
Put the English checkpoint in front of an English queue: refund or not, which desk, how urgent. Do not put this checkpoint on a multilingual queue. The publisher warns it can be confidently wrong on non-Latin scripts. The multilingual and typed-decision folders, and the publisher’s Router path, were not timed. Their ~33 ms card figure is a different harness.
Our number is one English checkpoint, one stream, one forward per call, on a fixed four-question sample email shipped with the bench. A wiring check on that sample came back billing, refund requested, churn language present. That checks the pipe. It is not an accuracy test, and it is not a customer message.
Allocated VRAM was about 2.3 GiB.
| Card | $/hr | p50 | Decisions/s | Decisions per $ |
|---|---|---|---|---|
| A6000 | 0.57 | 45.0 ms | 22.2 | 39.0 |
| L40S | 0.97 | 14.3 ms | 70.0 | 72.2 |
| Blackwell | 2.19 | 10.2 ms | 98.0 | 44.7 |
Decisions/s is forwards per second. Each forward classified one four-question ticket. It is not token throughput, and it is not four times the call rate. The A6000 wait is a bit more than 3× the L40S (45.0 ms versus 14.3 ms) and a bit more than 4× the Blackwell (45.0 ms versus 10.2 ms).
Screenshots and the raw capture: gpu-benchmark pull 38.
Launch a GPU after you check the live rate. The dollars above are the 2026-09-22 list, not a contract.
Deploy Which GPU for a 7 GB 27B, and which GPU for a ticket router
Launch NVIDIA L40 instances optimized for OpenAI-compatible inference. Get 48GB VRAM, blazing-fast NVMe storage, and per-second billing.
Think it. Build it. Scale it.
