E2A-Bench: Benchmarking Evidence-to-Action Reliability in Financial Chart Reasoning
Xiaoya Wang, Yutong Xu, Junjie Wang
Tsinghua University · Jinan University
Published September 13, 2026 · on the feed September 15, 2026 · ▲ 3 · arXiv 2609.14302
The abstract
Can financial vision-language models (VLMs) turn chart evidence into reliable action recommendations? Existing hallucination evaluations are mostly claim-centric; they assess whether generated statements are supported, but not whether evidence remains traceable through rationale, confidence, and final action. We introduce E2A-Bench, a 969-query benchmark for financial chart reasoning, constructed from 323 HS300 constituents under three input modalities with deterministic OHLCV-derived evidence anchors. E2A-Bench evaluates grounding, reasoning-action consistency, evidence-confidence calibration, and directional coverage through UCR, RCI, ECI, and NDR, where NDR measures coverage-aware evidence-to-action reliability rather than realized trading performance. Evaluating 20 VLMs reveals three failures hidden by scalar hallucination scores: the lowest-UCR model ranks near the bottom by NDR due to only 6.4% directional coverage; oracle-aided verification reduces unsupported claims but can collapse coverage; and financial fine-tuning amplifies the BUY:SELL ratio by factors of 4.21 to 4.68 across strict base-fine-tuned pairs. These results show that financial VLM evaluation should trace the full evidence-to-action chain rather than rely on a single hallucination score. Code and data: https://github.com/wanng-ide/E2A-Bench
The Brief
2:21A short podcast summary of the paper. Written and voiced by AI.
Transcript
E2A-Bench. Benchmarking Evidence-to-Action Reliability in Financial Chart Reasoning. By Xiaoya Wang and two others, from Tsinghua University and Jinan University. Published on the thirteenth of September. Its code repository is public but slight: six source files.
Can financial vision-language models turn chart evidence into action recommendations that can be relied on? Existing hallucination evaluations are mostly claim-centric. They assess whether generated statements are supported, but not whether the evidence remains traceable through the rest of the chain: through the rationale, through the stated confidence, and into the final action. A model could state only supported facts and still recommend an action its evidence does not back, or express confidence its evidence does not justify. Claim-level checks would see neither.
E2A-Bench measures four properties: grounding, consistency between reasoning and action, calibration between evidence and confidence, and directional coverage. These correspond to four metrics, UCR, RCI, ECI and NDR. NDR measures evidence-to-action reliability with coverage taken into account, and the authors are explicit that it does not measure realized trading performance.
The benchmark holds nine hundred sixty-nine queries on financial chart reasoning. They are built from three hundred twenty-three HS300 constituents, presented under three input modalities, and each query has deterministic evidence anchors derived from OHLCV data, so what the chart supports is fixed in advance. Because the anchors are computed deterministically from the OHLCV series, grading does not depend on anyone's reading of the chart.
Twenty VLMs were evaluated, and three failures surfaced that scalar hallucination scores hide. First, the lowest-UCR model ranks near the bottom by NDR, because its directional coverage is only six point four percent. Second, oracle-aided verification reduces unsupported claims but can collapse coverage. Third, financial fine-tuning amplifies the ratio of buy to sell recommendations by factors between four point two one and four point six eight, across strict pairs of base and fine-tuned models.
Each failure breaks a different link in the chain, and none of them shows up in a single hallucination number. The authors conclude that financial VLM evaluation should trace the full path from evidence to action rather than rely on one score. The code and data are public.
Paper Quest · 7 chapters · about 35 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A model that never answers can never be wrong. That is the trap in judging chart-reading AI.
You are judging AI models that read stock charts and recommend BUY, SELL, HOLD or UNCERTAIN, with a confidence level.
The road ahead
- 1 · The problem ~7 min Counting false statements cannot tell a careful stock-chart model from one that simply never recommends anything or recommends against its own reasoning.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Check every link from chart facts to final action against rule-computed facts, and let how often a model commits count in its score.
- 4 · Choose your own research ~3 min
- 5 · How it works ~3 min
- 6 · The experiment ~6 min You’ll guess: By what factor did financial fine-tuning amplify the ratio of BUY to SELL recommendations, across strict base and fine-tuned model pairs?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.