RiskChainBench: A Benchmark for Obfuscated Platform Message Restoration and Evidence-Grounded Web Investigation
ZhuoXin Liu, Zhiming Ma, Ying Zhang, Mengzheng Yang, Yifan Wang, Zhengqi Huang, Yanhan Zhou, Zekun Lin and 5 others
Baidu · SmartFlowAI · People's Public Security University of China · Tsinghua University · JD Technology · Northeastern University
Published September 15, 2026 · on the feed September 18, 2026 · ▲ 47 · arXiv 2609.16900
The abstract
Platform abuse campaigns conceal redirection instructions with emojis, homophones, character decomposition, and redundant symbols, then route users through disguised links to services associated with pornography, fraud, gambling, or illicit transactions. Existing benchmarks evaluate obfuscated text and risky webpages separately, obscuring how target recovery affects downstream evidence acquisition. We introduce RiskChainBench, pairing 3,600 synthetic token-text restoration inputs from 600 source sessions with 600 corresponding human-labeled local web environments. A model first restores the message, operational intent, and destination; the same underlying model then acts as a VLM-driven web agent that investigates the correctly associated website and produces a frozen, evidence-cited risk report without message-side semantics or domain-reputation cues. We score restoration and correct-routing web investigation separately and compose them offline by applying the frozen primary-entry prediction as a gate to the same Task 2 result. Human labels determine task correctness, while a fixed multimodal evidence judge assesses faithfulness, sufficiency, completeness, and consistency. Across ten models, Entry Top-1 ranges from 35.2% to 95.2% and web decision accuracy from 26.3% to 62.8%; the leading systems differ across entry recovery, full reconstruction, website decisions, and fine-grained typing. Execution failures account for 31.9% of web runs, whereas post-decision type errors account for only 0.9%, identifying stable exploration and risk judgment as the principal bottlenecks. We release the benchmark, protocol, and resettable local sandbox.
The Brief
2:45A short podcast summary of the paper. Written and voiced by AI.
Transcript
RiskChainBench. A Benchmark for Obfuscated Platform Message Restoration and Evidence-Grounded Web Investigation. By ZhuoXin Liu and twelve others, from Baidu, SmartFlow AI, and four other institutions. Published on the fifteenth of September. Its code repository is public and real: about thirty-five source files.
Platform abuse campaigns hide redirection instructions using emojis, homophones, split-apart characters and redundant symbols, then route users through disguised links to services tied to pornography, fraud, gambling or illicit transactions. Catching that involves two linked steps: decoding the hidden message, and then investigating where it leads. Existing benchmarks evaluate obfuscated text and risky webpages separately, which obscures how recovering the target affects the evidence gathered downstream.
RiskChainBench evaluates the chain end to end. A model first restores the message, its operational intent, and its destination. Then the same underlying model acts as a web agent driven by a vision-language model. It investigates the correctly associated website and produces a frozen risk report that cites evidence, with no access to message-side semantics or domain-reputation cues. The site has to be judged on what the agent finds there.
The benchmark pairs three thousand six hundred synthetic token-text restoration inputs, drawn from six hundred source sessions, with six hundred matching local web environments labelled by humans. Restoration and correct-routing web investigation are scored separately, then composed offline: the frozen primary-entry prediction acts as a gate on the same web investigation result. A model that decodes the message wrongly therefore gets no credit downstream, however well it might have investigated.
Human labels decide whether a task is correct. A fixed multimodal evidence judge assesses the report for faithfulness, sufficiency, completeness and consistency. The benchmark, protocol and a resettable local sandbox are released.
Ten models were tested. Entry Top-1 ranges from thirty-five point two to ninety-five point two percent. Web decision accuracy ranges from twenty-six point three to sixty-two point eight percent. The leading systems differ depending on the measure: entry recovery, full reconstruction, website decisions, and fine-grained typing each have their own best performer.
The failure analysis is pointed. Execution failures account for thirty-one point nine percent of web runs, while type errors after a decision account for only zero point nine percent. Models that get as far as a decision rarely mislabel the kind of risk. The principal bottlenecks are stable exploration and the risk judgement itself.
The benchmark's contribution is to keep the chain intact. Decoding and investigating are measured together, so a model that misreads the message is seen to fail at the website, and the weakest link in the chain can be located.
Paper Quest · 7 chapters · about 34 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Every quest starts with a wall. Here, abuse hidden in emojis and disguised links.
You moderate a platform. Abusers hide redirection instructions with emojis, homophones, split-apart characters and junk symbols, then send users through disguised links to gambling, fraud or worse.
The road ahead
- 1 · The problem ~6 min Abusers hide links in scrambled messages, and catching them takes decoding plus investigating, yet existing tests grade those two steps apart.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Make the model decode and commit first, then investigate the right site blind to the message, and count the verdict only if the decoding was right.
- 4 · Choose your own research ~3 min
- 5 · How it works ~3 min
- 6 · The experiment ~6 min You’ll guess: Across the web-investigation runs, which kind of failure was far more common: agents failing to carry out the exploration, or agents mislabelling the type of risk after deciding?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.