本文提出 V-RAGBench——一个由 ⟨query, evidence chunk, answer⟩ 三元组构成的基准,可对检索与生成进行忠实且解耦的评估;同时提出 CARVE,一种在多种配置下并行运行检索器、并通过分块自适应重排序为每个分块挑选最优配置的简易方法。V-RAGBench is introduced, a benchmark of $\langle$ query, evidence chunk, answer$\rangle$ triplets that enables faithful, decoupled evaluation of retrieval and generation, and CARVE, a simple method that runs parallel retrievers across configurations and employs chunk-adaptive reranking to identify the winning configuration for each chunk.
论文
13 张论文卡片 · RAG 检索增强 · 评测集
提出一个面向 RAG 的 AI 系统基准测试(RAGPerf)框架,用于刻画 RAG pipeline 的系统行为,并证明其引入的性能开销可忽略不计。The design and implementation of a RAG-based AI system benchmarking (RAGPerf) framework for characterizing the system behaviors of RAG pipelines is presented and it is shown that RAGPerf incurs negligible performance overhead.
MissDiag 将聚合鲁棒性度量转化为类型化的诊断归因,为在不完备知识下比较、诊断和压力测试 KGQA 与 KG-RAG 系统提供了更具可解释性的基础。By transforming aggregate robustness measurement into typed diagnostic attribution, MissDiag provides a more interpretable basis for comparing, diagnosing, and stress-testing KGQA and KG-RAG systems under incomplete knowledge.
基于语音的应用在接入检索模块之前需先通过自动语音识别(ASR)处理口头查询,因此 ASR 错误会以固定的上游约束进入 pipeline。我们通过实验验证标准检索增强生成(RAG)的两项扩展——实体图链接与迭代式 query 改写——是吸收还是放大了这些错误。基于神经 TTS 合成的四种英语口音,我们在三个多跳 QA 基准(HotpotQA、2WikiMultiHopQA 和 MuSiQue)上评测四种 RAG 配置,以干净文本 oracle 为对照。尽管结构上更丰富的 configuratioSpeech-based applications pass spoken queries through automatic speech recognition (ASR) before any retrieval module, so ASR errors enter the pipeline as a fixed upstream constraint. We empirically test whether two extensions to standard retrieval-augmented generation (RAG), entity-graph linking and iterative reformulation, absorb or amplify these errors. Using four English accents synthesized through neural TTS, we evaluate four RAG configurations on three multi-hop QA benchmarks (HotpotQA, 2WikiMultiHopQA and MuSiQue) against a clean-text oracle. Although the structurally richer configuratio
基于实证对 "Enhanced" 与 "Agentic" RAG 范式进行评估,为真实场景中选取最有效的 RAG 设计(兼顾性能与成本)提供指导。An empirically driven evaluation of the "Enhanced" and "Agentic" RAG paradigms is conducted, offering guidance on selecting the most effective RAG design for real-world applications, considering both performance and costs.
将预测的味觉空间作为基于内容的检索索引,对 309 项条目池的排序比 CLAP-text 基线(处于随机水平)忠实得多;ridge probes 与 audio-bandstop knockout 在已记载的声-味对应关系上读出了最强表征。Operationalised as a content-based retrieval index, the predicted taste space ranks a 309-item pool far more faithfully than a CLAP-text baseline, which sits at chance; ridge probes and an audio-bandstop knockout read the strongest representations against documented sound-taste correspondences.
提出 MIRROR——一个统一的跨表层框架,在显式新颖性约束下以检索到的上下文为条件生成候选,并执行记忆引导的蒙特卡洛树搜索,使检索可影响搜索先验,同时避免提示词级别的复制。MIRROR is presented, a unified cross-surface framework that performs memory-guided Monte Carlo tree search while conditioning candidate generation on retrieved context under an explicit novelty constraint, allowing retrieval to inform search priors without enabling prompt copying.
HAKARI-Bench 是一个轻量级基准,将现有检索套件重建为统一格式的小型数据集(Nano-sets),支持在同一条件下对五类检索方法及其效率变体进行与模型无关的对比。HAKARI-Bench is a lightweight benchmark that reconstructs existing retrieval suites into small datasets (Nano-sets) in a unified format, enabling same-condition, model-agnostic comparison of five retrieval families and their efficiency variants.
本文命名并测量了推测性查询的检索收敛到包含答案结果时的输入流位置——"工具意图稳定化":即推测性查询的检索收敛到包含答案结果的输入流位置。This work names and measures the point in the input stream at which a speculative query's retrieval converges on the answer-bearing result, tool-intent stabilization: the point in the input stream at which a speculative query's retrieval converges on the answer-bearing result.
对一个旨在使魁北克汽车保险合同更易理解的SOTA RAG系统进行以人为中心的外在评估,结果显示该系统被视为认知均衡器,用户对系统所提供的自主感的重视程度甚至超过知识本身。A human-centric, extrinsic evaluation of a state-of-the-art Retrieval-Augmented Generation system, designed to make Quebec automobile insurance contracts more understandable, shows the system is perceived as a cognitive equalizer, and users value the sense of autonomy the system provides even more than the knowledge itself.
提出知识对齐的多跳检索器 KAMR,区分受 query 强约束的 anchor triplet 和弱对齐但在结构上与 anchor 相连的 connected triplet,持续提升多跳检索及下游问答性能。A knowledge-aligned multi-hop retriever, KAMR, which distinguishes anchor triplets that are strongly constrained by the query from connected triplets that are weakly aligned yet structurally linked to the anchors, which consistently improves multi-hop retrieval and downstream question answering performance.
研究表明在希腊语专业语料上,无参数 BM25 基线优于多个现成的多语言稠密检索模型,并推出首个大规模希腊语 RAG 基准 HERA,同时发布适配模型与基准以支撑未来希腊语 RAG 系统研究。This study shows that a parameter-free BM25 baseline outperforms several off-the-shelf multilingual dense retrieval models on specialist Greek corpora and introduces HERA, the first large-scale Greek benchmark for retrieval-augmented generation, and releases adapted models and benchmark to support future research on Greek-language RAG systems.
提出 GRIP(Grounded Reasoning via Information-Restricted Premises),引入容量不对称:decoder 对 query 保持全维度访问,而检索到的证据则通过一个严苛的随机瓶颈,迫使证据通道仅编码 query 中无法获得的残余信息。GRIP (Grounded Reasoning via Information-Restricted Premises), which imposes capacity asymmetry: the decoder keeps full-dimensional access to the query, while retrieved evidence passes through a severe stochastic bottleneck, which forces the evidence channel to encode only the residual information unavailable from the query.