FinRCA-Bench: Benchmarking Evidence Retrieval and Reasoning for Financial AI Systems

arXiv:2608.185342026-08-20

AI经常能说对财务对账出错的原因,却拿不出真正的证据

一个名为FinRCA-Bench的新基准专门测试AI系统能否准确诊断发票、付款、银行记录之间对不上账的原因。研究人员只更换检索证据的方法、保持AI模型本身不变,结果准确率就从2.05%飙升到72.44%,说明起决定作用的是能否找到正确的证据记录,而不是模型本身的推理能力。即便答对了,严格满足证据要求的比例也只有5.72%。

论文原文摘要（英文）

Large language models are increasingly used to support financial operations, but their apparent reasoning performance can depend on whether they receive the right evidence. In financial reconciliation, the evidence needed for diagnosis is distributed across invoices, purchase orders, approvals, allocations, payments, ledger entries, and bank activity, linked by transactional relationships rather than textual similarity. End-to-end accuracy can ther

作者 · Pratik Ghawate