Asking for What Was Never Requested: Horizontal and Vertical Proactivity in Agents

Asking for What Was Never Requested. Horizontal and Vertical Proactivity in Agents. By Ido Levy and four others, from IBM and the Weizmann Institute of Science. Published on the twenty-ninth of September. Its code repository is worked out: about eight hundred source files, with a readme and a licence.

An agent that uses tools typically responds to what the user explicitly asks. Completing the task, though, may require information the user never requested. Work on proactive agents has mainly studied whether, and when, an agent should act on its own initiative. It has said much less about what information the agent should pursue. The authors take up that distinct axis: the content of proactivity. Deciding to speak up is one thing. Knowing what to ask about is another, and an agent can be eager without being useful if it pursues the wrong things.

They divide it in two. Horizontal proactivity pursues unstated information that the current context already identifies. The gap is visible from where the agent stands. Vertical proactivity pursues needs that only earlier evidence reveals. Those needs come into view only once something else has been found. The first kind widens the search at the current step. The second deepens it, following one finding to the next question it raises.

To score both, the authors recover a need graph from a benchmark's own decomposition. The graph records which needs depend on which. With it, both forms of proactivity can be scored directly from a transcript, and so can whether the agent stops at the right time, all without a model acting as judge. Stopping is part of the score on purpose. An agent that keeps asking after the needs are met is not being proactive, only expensive.

To learn the behaviour, they propose Q and D, short for questioner and drafter. The questioner is trained to prefer the question whose continuation retrieves more of the required evidence. There is no reward model and no judge in that training. The signal is simply how much of what the task needed a question ends up bringing back. A question is good, on this view, if asking it leads toward the evidence the task required.

The first tests use held-out splits of three multi-hop question-answering benchmarks, at equal retrieval spend. There, the trained questioner improves both forms of proactivity over the same model when it is only prompted. On two of the three benchmarks, it also outperforms a prompted model fifteen times larger in the same role. That gain persists after controlling for how many questions are asked and how long they are.

Next, without any further training, the questioner is placed inside an interactive customer-service agent that faces a simulated customer. It completes more tasks while asking fewer questions. In the retail setting, it outperforms the fifteen times larger model, and the customer needs fewer follow-up turns.

The usual question about a proactive agent is whether it acts without being asked. These results add two more: what it chooses to pursue, and when it stops. A trained questioner that asked fewer questions still completed more tasks, and in the authors' reading that is the point. Proactivity is a matter of content as well as initiative.