Matched Capabilities, Different Behaviors: A Controlled Study of MCP and CLI Interfaces for Language Agents

Matched Capabilities, Different Behaviors. A Controlled Study of MCP and CLI Interfaces for Language Agents. By Zhenning Shi and two others, from the Hong Kong Polytechnic University and the Institute of Computing Technology of the Chinese Academy of Sciences. Published on the twenty-fifth of October.

Language agents increasingly act through external tools and interactive environments, which makes the interface between a model and what it can execute part of the decision process itself. The Model Context Protocol and command-line interfaces can expose the very same semantic operations, but present them to the model in markedly different ways. The question is whether that difference in presentation alone changes how an agent behaves, once the underlying capabilities are matched. The answer is that it does, substantially.

To isolate the effect, the authors build MICA, short for Matched Interface Comparison and Analysis. It renders both the MCP and the CLI interface from one shared semantic action contract, and routes the decoded actions through the same runtime. Across six models and three interactive environments, the matched interfaces produce substantial differences in task performance and in interaction cost. The effects go beyond whether actions are superficially valid, reaching which semantic actions get chosen, how failures unfold, and how agents recover.

Probes from matched states localize the sensitivity to three places: how a capability is presented, how it is realized, and what feedback comes back. Paired natural trajectories show structured divergence in ninety-four point three percent of two thousand and four episodes, and early differences carry forward through the rest of the interaction.

That analysis points to interface realization as a direct place to intervene. DART, Decoupled Action Realization and Targeted Repair, keeps the semantic commitments that have been validated and repairs only the realization specific to the interface. It improves benchmark score in all six evaluated settings, by up to nine points, and reduces MCP token cost in all three environments, by up to twenty-seven point one thousand tokens per task.

The comparison covers two interface styles, MCP and CLI, across six models and three interactive environments, with capabilities matched by construction through a shared action contract and a shared runtime. The claim concerns representation with capability held fixed. It is not that one interface can do more than the other, since by design neither can. It is that the same operations, presented differently, lead the same model to act differently. The repair results are bounded by the same six settings, and the token savings are reported for the MCP side.

When two interfaces offer the same actions, an agent still does not treat them as the same. The authors' conclusion is that interface representation belongs to the closed-loop decision environment that shapes agent behaviour, rather than sitting outside it as a neutral wrapper.

Matched Capabilities, Different Behaviors. A Controlled Study of MCP and CLI Interfaces for Language Agents. By Zhenning Shi and two others, from the Hong Kong Polytechnic University and the Institute of Computing Technology of the Chinese Academy of Sciences. Published on the twenty-fifth of October. This is the long version: the whole paper rather than the abstract.

An agent that acts through tools can be handed the same capabilities in two very different costumes. The Model Context Protocol shows each capability as a named tool with a structured argument object. A command-line interface shows it as a command with flags and quoted text. Existing comparisons tend to change other things at the same time, such as tool granularity, prompting or scaffolding, so nobody can say whether a gap comes from the representation or from what the agent was actually allowed to do. The second problem is that a final score hides the route. The same number can come from different choices, different failures and different costs, and one early difference can rewrite everything that follows.

The paper's answer is a framework called MICA, for Matched Interface Comparison and Analysis. It starts from a single semantic action contract per benchmark: a list of operations, their typed arguments, the values those arguments may take in the current state, and how each one compiles to a native environment command. Both interfaces are rendered from that one contract. Whatever the model writes is decoded back into the same normalized action, then passed through the same validator, the same compiler and the same executor. In a household environment, a tool call naming go with a destination of living room and a command reading go, dash dash destination, living room, are the same action. Only what the model sees and types differs.

MICA then splits an interface into three parts. Presentation is how the capabilities and constraints are described to the model. Realization is the response grammar plus the decoder that parses it. Feedback is how results and error messages are rendered back. Because the parts are separable, they can be mixed: MCP presentation with command-line realization, and so on, giving eight conditions. These are matched-state probes. A real decision point from a recorded episode is frozen, the state and history restored exactly, and the model is sampled eight times under each of the eight conditions. That covers three hundred and thirty-eight source states and twenty-one thousand six hundred and thirty-two sampled responses.

The second scale is the whole trajectory. Paired episodes start from the same task and state. At every step the paper records an interaction signature: the decoded action, where it failed if it failed, the execution outcome, and the recovery status. The first step at which the two signatures disagree is the first consequential divergence. Surface strings may differ before that point, as long as they decode to the same action and lead to the same outcome.

The repair method, DART, stands for Decoupled Action Realization and Targeted Repair. It only activates after a retryable failure that left the environment unchanged. The same model is first asked to choose an action as plain structured data, an operation and its arguments, with no interface syntax at all. If that choice passes the shared validator, it becomes a commitment tied to the current contract. A second call asks the model to express exactly that commitment in MCP or command-line form. The expression is accepted only if it decodes back to the committed action and is still valid. If it fails, one correction is allowed, which may change quoting or structure but not the operation or any argument. Otherwise control falls back to the ordinary retry.

The endpoint results refuse to pick a winner. For the fourteen-billion- parameter Qwen model, MCP trails the command line by thirty-six point six points on ALFWorld, twenty point one against fifty-six point seven, yet leads on WebShop. The appendix shows why the ALFWorld cost went up: calls per task rose from forty-six point nine to fifty-seven point one while tokens per call barely moved. The episodes simply ran longer. Even among responses that decoded cleanly, the mix of chosen actions shifted between interfaces.

The probes test the thesis most directly. Realization dominates. For the thirty-two-billion-parameter Qwen model on ALFWorld, swapping only the realization format moved realization failure from one percent to eighty-seven percent under MCP presentation. Interactions are large too: one three-way interaction shifted admission by thirty-seven point eight points. Across two thousand and four paired episodes, ninety-four point three percent diverged, mostly within the first few steps, and mostly through a different action rather than a different spelling. DART, tested on Gemini two point five Flash, raised score in all six benchmark and interface settings, by up to nine points, and cut MCP token cost in every environment, by up to twenty- seven point one thousand tokens per task.

The paper has no limitations section. Its evidence still has visible edges. DART was evaluated on one model only, Gemini two point five Flash, so the claim that it helps across models is not tested. Its gains vary in size, as low as under one point on WebShop with the command line, and under the command line its token cost rose in two of the three environments. The main results table reports no intervals, although several appendix figures carry bootstrap intervals over tasks. Natural episodes were decoded at temperature zero. And all three environments are text benchmarks with a purpose-built command surface, not live MCP servers or real shells, so how far the effect sizes carry to deployed agents is not shown.

The idea to carry away is that an interface is not a passive pipe. Hold the capabilities exactly fixed and the agent still chooses different actions, fails at different stages, recovers differently, and ends up on a different path, usually from its first few moves. The practical lesson from DART is narrow and useful: when an action fails only in how it was written, keep the decision and repair the expression.