A file is not an engineproject

My model sweep runs every two weeks to answer one question: has anything shipped that could unseat what I’m running? Two weeks after the new GPU settled in, it finally fired. The candidate was Qwen3.8-27B, the same model already holding the local seat behind my agents, quantized by a different account, with one difference that mattered: this version kept the model’s MTP head intact. Multi-token prediction lets the model draft a few tokens ahead and verify them in one pass, and reports I’d been tracking put the speedup at 50% or more. I was already sold on the model. Everything around it was guilty until being proven innocent.

The complication was the engine. Serving MTP on vLLM, the serving engine I run, had a known correctness concern in exactly my configuration, with the upstream fix stalled since spring. The model card had an answer ready: the artifact had been validated end to end on a fork of the serving engine, built to keep older GPUs alive, and the fork was there for the downloading.

Keeping older hardware alive is real work, and the artifact is genuinely good. What mattered to me was the difference between loading the weights and running the fork. The weights arrived as two safetensors files, a format the loader reads as a header plus raw tensor bytes, and stock vLLM served them without the flag that lets a repository run its own model code. Serving this artifact never required executing the fork or anything from the model’s repository. A fork is code, and code I run on the homelab gets GPU access, filesystem mounts, and a network stack the moment it starts. I am deliberate about how much of other people’s work I adopt versus validate with my own tools. Download counts and stars are proxy stats, and I don’t treat either as a marker of trustworthiness. The weights would have to earn their place with my own harness.

The harness was built around one property. Speculative decoding checks every drafted token against the full model, so at temperature zero its output is supposed to match the plain path, and corruption should show up as a diff you can read. I ran the candidate with MTP on and off across my agents’ real workloads, 60 prompts one at a time and six batches of eight mixed requests, plus a control arm that ran the plain path against itself, because ordinary batch wobble had fooled me once before. Generic benchmarks assume someone else’s use case, hardware, and definition of success, and mine skew obscure and highly personalized, so this gauntlet is one of my own making.

The premise did not survive the control arm. The plain path, run twice against itself, disagreed on 20 of 48 batched rows. This model sits on a lot of near-ties, and my best explanation is numerical: batching changes the arithmetic just enough to tip a near-tie either way, and speculative verification changes the shape of the forward pass too. Exact agreement with the plain path was the wrong bar. After the first window, on evidence that had nothing to do with the candidate, I replaced it with three checks.

It passed those checks on those runs, which is the strength of claim the harness can make. The weights went to production that night. Live decode measured 99 tokens per second against about 64 without the head, and because this artifact’s memory floor came in about 5 GiB smaller, the serving cache had room for roughly twice as many tokens.

The artifact held the production seat for seventeen days. A mid-September sweep replaced it with a different quantization of the same model, MTP head intact, and that candidate went through the same diff harness on its way in; every divergence it found looked like one of those tipped near-ties, with no repeated or truncated output. The engine under both is still stock vLLM, updated in place by Patchday. Two artifacts in, the fork has still never run on my homelab.