The Other 5000

It started with an eBay seller I’d bought from before, and a number: 5000.

I’ve got two refurbished Dell Precision 5820 towers, both bought cheap — the kind of cheap that happens when a company refreshes its workstations and the recycler just wants them gone. One came with an RTX 2080. The other, luckier, came with a Quadro RTX 5000.

The good 5000#

That Quadro RTX 5000 had been quietly carrying me. Sixteen gigs of GDDR6, Turing silicon, the good stuff. I’d been running qwen3.6-27b on it at a 3-bit quant — squeezed in tight — and it wasn’t half bad. Decoding at 15–20 tokens a second, fast enough to keep me in the loop. I pointed it at some genuinely nasty bugs in a project I was wrestling with, and it earned its keep: it caught things I’d stopped being able to see because I’d been staring too long.

So naturally I wanted more. A pair of 5000s. Split the tensors across two cards, run something bigger, stop bumping the ceiling. I went looking.

The other 5000#

The reputable seller had Quadro P5000s listed. P5000. I read “5000” and filled in the rest myself.

This is the part where a single letter does a lot of work. The Quadro RTX 5000 is Turing — 2018, modern tensor cores, compute 7.5. The Quadro P5000 is Pascal — 2016, compute 6.1, a card that is, as I write this, roughly a decade old. Same “5000.” Same 16GB framebuffer, even. Not the same card. Not the same decade.

Honestly, NVIDIA: how many different 5000s does one number need? RTX 5000, P5000, and — I checked — there’s an M5000 too, just in case the chaos needed a mobile edition.

Stated plainly, because it’s worth knowing before you buy: two years is all that separated these two. October 2016 to late 2018. But the RTX 5000 ships with Tensor Cores, and the P5000 has none. Not a few. Zero.

That gap matters more than the calendar suggests. An LLM is, at the bottom, a stack of matrix multiplications. Tensor cores are the hardware built to do exactly that — fused multiply-accumulate on small tiles, in FP16 or INT8, many lanes in a single beat. Take them away and the same math falls back to the ordinary CUDA cores, which will still do it, just much, much slower. Same job. Very different afternoon.

By the time I’d noticed, two of them were already on my bench. About $500 for the pair, which is the kind of number that makes you feel clever right up until it makes you feel stupid. I sat with them for a while. They were cheap. They were here. I couldn’t quite bring myself to list them again.

The way through#

Here’s the thing about the Precision 5820, though. The one that had been wearing the RTX 2080 had the big power supply — 950 watts — and a Xeon W with 64GB of RAM underneath it. That PSU had been loafing. Two P5000s at 180W each — it wouldn’t break a sweat.

So I pulled the 2080, dropped both P5000s in, and started over.

The container was the familiar one — my llama.cpp CUDA image — but I turned MMQ on. MMQ is the integer matrix-multiply path — the quantized GEMM kernels that predate tensor cores entirely. The modern fast paths want newer silicon; Pascal has to fall back on the old one.

The remarkable part is that it runs at all, and that’s not luck. The people maintaining llama.cpp have kept the MMQ kernels alive on purpose, so compute 6.1 cards like mine don’t just age out of the stack. Nobody made them. There’s no glory in Pascal. They did it anyway, and this tower is the proof: a decade-old card with no tensor cores to its name, holding 16 tok/s on a 27B model.

Tensor-split down the middle, 50,50. K/V cache in q8_0 to buy back some VRAM. Flash attention on. The Unsloth Dynamic UD-Q4_K_XL quants, specifically, because they’re the ones that keep the MTP heads intact — and MTP heads mean speculative decoding, two draft tokens at a time.

A pair of ten-year-old cards, in a salvaged tower, in a throwaway container. That was the rig.

The numbers#

I ran my eval suite against it. The Pascal pair did not embarrass itself.

Graders, world knowledge, tool calling, reasoning, path precision, an 80K needle-in-a-haystack — both clear about 95% of it. And the MoE is the stunner: same accuracy, two and a half times the speed, three times faster on the long-context needle. It even fixed a bug the dense model missed. Max stable context: 192K. On two cards from 2016.

Here are the presets behind those numbers. The two models share almost everything; the MoE just runs a shorter context:

# llama-server --models-preset models.ini
[qwen3.6-27b]
model            = /models/Qwen3.6-27B-UD-Q4_K_XL.gguf
ctx-size         = 196608
tensor-split     = 50,50
n-gpu-layers     = 99
cache-type-k     = q8_0
cache-type-v     = q8_0
flash-attn       = on
spec-type        = draft-mtp
spec-draft-n-max = 2
temp             = 0.6
top-p            = 0.9
min-p            = 0.05
repeat-penalty   = 1.05
seed             = 3407

# [qwen3.6-35b-a3b]: same knobs, different model, ctx-size = 131072

Grab models.ini.sample from docker-llamacpp-cuda — the image is already built, so you’re a docker run away from the same rig.

The catches#

The bench passes. Live coding is where the seams show.

First, the low quants lie about file paths. At 4-bit the model knows a path should be there and happily invents one that looks right and isn’t. Bump the dense 27B to 5- or 6-bit and the hallucination dries up. The path-precision test passes in the suite; real repos with real relative paths are meaner.

Second, the harness. I’d been running @badlogic’s pi, and the latest pi.dev does not love these models. The edit tool calls go sideways — partial patches, mangled hunks, the model reaching for a tool it can’t quite steer. The fix wasn’t in the model and it wasn’t in pi. It was a guardrail: an AGENTS.md that tells the agent how to behave on this hardware, with this model. Constrain the edges and the whole thing smooths out.

These days#

It’s driving my agent now — nanobot on a Gemma 4 model — and after some twiddling it does what I ask and gives me factual answers. This one even sees: it’s a vision model, so it takes images alongside text. For fun I fed it a photo of some Mongolian paper money I had in a drawer, and it read the note right back to me — script, denomination, the lot. I’m not Mongolian, for the record — my young friend Deggy is, and he’s the one who gave me the note. Nothing leaves the room, no token meter, no rate limit. Two cards I almost regretted buying.

A mistake isn’t necessarily a bad thing. The “good” 5000 is still in the other tower, still running. The two I didn’t mean to buy became the rig I actually use. Turns out the matrix has more than one door — and some of the best ones you back into by accident.