SiliconBench: Speed, Memory, and Fidelity for LLM Serving on Unified-Memory Desktops

SiliconBench. Speed, Memory, and Fidelity for LLM Serving on Unified-Memory Desktops. By Ranran Haoran Zhang and four others, from Penn State University, the University of Illinois Urbana-Champaign, and one other institution. Published on the twelfth of September. Its code repository is worked out: about one hundred source files, with a readme and a licence.

Serving local language models concurrently on a unified-memory desktop is a different job from serving them one request at a time. Several requests share one pool of memory, and the engine has to preserve headroom in that pool while still returning faithful output. Rankings by speed alone overlook both of those requirements. An engine can top a throughput chart while running the machine to the edge of its memory, or while quietly returning worse answers.

SiliconBench evaluates nine Apple Silicon serving engines through three lenses: speed, memory, and fidelity. It covers two kinds of serving, chat and agent, on three model families: Qwen3, Qwen3.5, and Gemma 4.

Fidelity is checked with a classification task, looking for quality regressions against an NVIDIA reference, so any drop in quality introduced by an engine shows up as disagreement with that reference. DGX Spark serves as a complementary reference for serving performance. Three desiderata guide how the results are read. The first is whether the serving architecture is ready for concurrent use. The second is memory discipline. The third is scaling across multiple nodes.

On Qwen3-0.6B, vllm-metal alone more than doubles throughput on both workloads between concurrency one and concurrency sixteen. CUDA vLLM and SGLang show stronger concurrency scaling on the same prompts. Explicit memory budgets do not guarantee headroom: two stacks complete every request while their memory use approaches physical capacity and their throughput declines. The newer model architectures have narrower engine support, although the implementations evaluated do match the fidelity reference. Only three stacks pass all three gates, for completion, fidelity, and model coverage. That last result is the one the three lenses exist to produce. Of nine engines, most fall short on at least one of the three, which is exactly what a ranking by speed alone would not reveal.

Comparisons on larger dense and mixture-of-experts models reinforce the importance of scheduling prompt processing alongside ongoing generation. Vllm- metal's packed prefill-decode path keeps first-token latency lower than omlx under concurrent load. Handling new prompts while other requests are still generating is, on this evidence, where engines separate once load arrives. And across two machines, in the configurations tested, tensor parallelism over Thunderbolt RDMA scales, while pipeline parallelism over TCP regresses.

The release covers the benchmark code, per-run results, and maintenance journals, kept up by a workflow that pairs bounded agent fixes with human review. The numbers carry one message. On a desktop that shares its memory, a fast engine is not yet a usable one: it also has to leave room, and it has to keep giving the right answers.