VākQA: A Benchmark and Evaluation Study for Telugu Spoken Factoid Question Answering

  • 类型：arxiv
  • 标识：2609.19879
  • 链接：https://arxiv.org/abs/2609.19879
  • 主分类：evaluation
  • 形态：benchmark
  • TLDR：Question answering has advanced rapidly with large language models, but predominantly for high-resource languages, in both text and spoken settings. Spoken question answering (SQA) benchmark for Telugu remains unexplored, and the reliability of automatic evaluation in this setting remains unquantified. We introduce VākQA, a Telugu SQA benchmark of 2,001 factoid question-answer pairs across six domains, with 2.53 hours of speech audio, bilingual transcriptions, and human-verified reference answers. We first validate evaluation methods against human judgements: Gemini-as-a-judge best approximate
  • 副分类：multimodal
  • 待LLM分类：否
  • 标题中文：VākQA：泰卢固语口语事实型问答基准与评估研究
  • TLDR中文：大语言模型推动了问答技术的快速发展，但主要集中于高资源语言，无论文本还是口语场景。泰卢固语口语问答（SQA）基准仍是空白，且该场景下自动评估的可靠性尚未量化。我们推出 VākQA——一个包含 2,001 个事实型问答对、涵盖六大领域的泰卢固语 SQA 基准，配有 2.53 小时语音音频、双语转写及人工核验参考答案。我们首先将各评估方法与人类判断进行对齐验证：Gemini-as-a-judge 最佳逼近……
  • 来源文件：
  • /inbox/tom/_candidates/2026-09-18-agent-rag-longcontext-candidates.json