MAServeBench: Benchmarking Multi-Agent LLM Serving Systems with Hierarchical Collaboration Workloads

MAServeBench. Benchmarking Multi-Agent LLM Serving Systems with Hierarchical Collaboration Workloads. By Shengxuan Qiu and six others, from Peking University and the Georgia Institute of Technology. Published on the twenty-ninth of October.

Multi-agent systems built on large language models coordinate specialized agents to solve complex tasks. One user request becomes a web of interdependent model calls and tool operations. Unlike independent requests, those operations become ready at different times, and they consume context that keeps changing as agents coordinate and pass results to each other. The result is shifting patterns of concurrency, computation and memory demand, and those patterns can expose different bottlenecks in serving hardware. To benchmark that properly, it is not enough to record the requests seen during one execution. You have to capture the collaboration rules that generate the workload in the first place.

MAServeBench treats multi-agent collaboration as a configurable dimension of the serving workload, for evaluating both serving systems and hardware. Its task-linked measurements connect each workload to serving execution and to the behaviour of GPU and NPU hardware, so that efficiency at the serving layer can be set against whether tasks actually finish on time.

The benchmark is built from composable stages. Each captures how agents coordinate and exchange context in documented multi-agent workflows. Because the collaboration pattern is the configurable part, native executions can generate different workloads even when the task inputs and the tools stay the same.

For reproducibility it records Hierarchical Semantic Traces. These preserve the operations that were realized and the dependencies between them, which allows reproducible comparisons of legal release constraints while the amount of work is held fixed.

The finding the abstract highlights is a mismatch. Linking the workloads to serving execution and hardware behaviour reveals why serving efficiency does not always translate into timely task completion. A system can serve requests efficiently and still leave the multi-agent task waiting.

The shift MAServeBench asks for is in what counts as a workload. Instead of a fixed list of requests, it is the collaboration that produces them, made configurable and traced with its dependencies intact. Measured that way, the question for serving hardware moves from how fast it handles requests to how soon the whole task is done.