InternReviewer & InternAdvocate: Objective Reward and Evaluation for Agentic Reinforcement Learning in Peer Review and Rebuttal
Xuerui Su, Liya Guo, Qizhi Pei, Qipeng Guo, Zhongbo Tian, Lijun Wu, Kai Chen, Zun Wang
Shanghai AI Laboratory · Beijing Jiaotong University · Tsinghua University · Renmin University of China
Published July 21, 2026 · on the feed September 2, 2026 · ▲ 8 · arXiv 2608.28612
The abstract
Generating professional scholarly content, such as peer reviews and rebuttals, requires an intricate synergy between domain reasoning and factual grounding. This work presents a comprehensive framework for the development and evaluation of specialized scholarly agents, InternReviewer and InternAdvocate. We first establish a large-scale, high-quality scholarly dataset and integrate a high-efficiency arXiv retrieval tool to enable active evidence gathering. To optimize these agents, we implement an agentic Reinforcement Learning (RL) paradigm driven by a unified objective metric and reward system. This system avoids the biases of subjective model-based judging by employing multi-dimensional criteria, including reference-anchored semantic alignment, structural compliance, and a strict verification mechanism that cross-checks citations against real-time interaction logs to eliminate hallucinations. Experimental results demonstrate that agents trained within this closed-loop framework exhibit significant improvements in reasoning depth and citation accuracy.
The Brief
1:53A short podcast summary of the paper. Written and voiced by AI.
Transcript
InternReviewer & InternAdvocate. Objective Reward and Evaluation for Agentic Reinforcement Learning in Peer Review and Rebuttal. By Xuerui Su and seven others, from the Shanghai AI Laboratory, Beijing Jiaotong University, and two other institutions. Published on the twenty-first of July. No code repository is linked to it.
Writing professional scholarly content, peer reviews and rebuttals in particular, takes two things working together: reasoning within a domain, and grounding in fact. A review can reason well and still cite things that are not there. Training agents for this work has a further difficulty, which is how to score them, because subjective model-based judging brings biases of its own.
The work presents a framework for developing and evaluating two specialised scholarly agents, InternReviewer and InternAdvocate. They are optimized with an agentic reinforcement learning paradigm driven by one unified objective metric and reward system, designed to avoid those biases by replacing subjective judging with multi-dimensional criteria.
Three pieces hold it up. First, a large-scale, high-quality scholarly dataset. Second, a high-efficiency arXiv retrieval tool, so the agents can gather evidence actively. Third, the reward criteria themselves: reference-anchored semantic alignment, structural compliance, and a strict verification mechanism that cross-checks citations against real-time interaction logs. That last check is aimed squarely at hallucination, since a citation is compared against the record of what the agent actually did.
Agents trained inside this closed-loop framework show significant improvements in reasoning depth and in citation accuracy. The abstract does not give figures, so the size of those gains cannot be stated here.
The design choice that carries the work is the reward. Instead of asking another model whether a review is good, it scores what can be checked: agreement with references, adherence to structure, and whether each citation holds up against the log.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A reviewer that invents papers, graded by a judge that cannot agree with itself.
You are training an agent to write peer reviews with reinforcement learning. Plain language models invent technical flaws and cite papers that do not exist, which is unfair to authors.
The road ahead
- 1 · The problem ~7 min Training a review-writing agent needs a reward, and a strong model grading each review gives scores too inconsistent to train on.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Replace the subjective model grader with several objective, checkable reward criteria, including citations checked against the agent's real search logs.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Collect real papers with reviews and rebuttals, give the agents a fast arXiv search, define checkable rewards, then train with agentic reinforcement learning.
- 6 · The experiment ~6 min You’ll guess: Agents trained inside this closed loop of objective rewards and live search improved most clearly in which two qualities?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.