Papers · organized/paper_cards

论文

30 张论文卡片 · 多模态 · 评测集

开放获取 全部 绿色 · 677
SemComp-Bench: Benchmarking Semantic Task Completion in Video Generation
SemComp-Bench:视频生成中的语义任务完成度基准评测
arXiv:2608.17426 多模态 评测集 被引 0 · S2

在代表性视频生成模型上的实验表明,在保持参考图像中任务相关语义一致性的同时实现预期结果仍然具有挑战性Experiments on representative video generation models show that achieving intended outcomes while maintaining task-relevant semantic grounding in reference images remains challenging.

Hydra-0: Action Flow for Generalist World Modeling and Control
Hydra-0:面向通用世界建模与控制的动作流
arXiv:2608.18077 多模态 评测集

我们提出 Hydra-0,一种以动作流为条件的通用世界模型,将机器人动作表示为像素运动。这种共享的视觉接口使得跨具身、任务、环境和视频生成 backbone 的通用世界建模与控制成为可能,学习动作在不同场景下的后果。我们的最佳配置相比动作条件 baseline,机器人运动误差降低 90.4%,物体运动误差降低 60.2%,同时支持零样本组合与数据高效适配。在 RoboLab 基准上,Hydra-0 在 replayeWe introduce Hydra-0, a generalist world model conditioned on action flow, which represents robot actions as pixel motion. This shared visual interface enables generalist world modeling and control by learning action consequences across embodiments, tasks, environments, and video-generation backbones. Our best configuration achieves 90.4% lower robot-motion error and 60.2% lower object-motion error than our action-conditioned baseline, while supporting zero-shot composition and data-efficient adaptation. On the RoboLab benchmark, Hydra-0 achieves a Pearson correlation of r=0.96 between replaye

Multimodal 补充候选
arXiv:2508.17398 多模态 评测集 被引 3 · S2

本文提出 DashboardQA,这是首个明确设计用于评估视觉-语言 GUI Agent 对真实世界仪表板理解与交互能力的基准,结果表明交互式仪表板推理对所有受评估的 VLM 而言都是一项具有挑战性的任务。DashboardQA is introduced, the first benchmark explicitly designed to assess how vision-language GUI agents comprehend and interact with real-world dashboards, and indicates that interactive dashboard reasoning is a challenging task overall for all the VLMs evaluated.

DataComp-VLM: Improved Open Datasets for Vision-Language Models
DataComp-VLM:面向视觉-语言模型的改进开源数据集
arXiv:2606.28551 多模态 评测集 MPG.PuRe (Max Planck Society) OA · 绿色 被引 1 · S2

数据混合(而非过滤)是构建高质量训练数据集的关键:以指令型数据为主的混合在扩展时优于以描述型数据为主的混合,且规模越大优势越明显。It is found that data mixing, not filtering, is key to a high-quality training dataset: instruction-heavy mixtures scale better than caption-heavy ones, with gains widening at larger scales.

Discrete Diffusion Language Models for Interactive Radiology Report Drafting
用于交互式放射学报告起草的离散扩散语言模型
arXiv:2607.01436 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

本文适配了一款专家混合扩散语言模型 DiffusionGemma-26B,并在医学视觉问答数据集上,使用相同的 LoRA 配置将其与同规模的自回归模型 Gemma-4-26B 进行基准对比,由对冗长度鲁棒的 LLM 裁判打分。This work adapts a mixture-of-experts diffusion language model, DiffusionGemma-26B, and benchmark it against its same-size AR sibling Gemma-4-26B under an identical LoRA recipe on medical visual question answering datasets, scored by a verbosity-robust LLM judge.

Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning
打破失败级联:面向医学多模态推理的步骤感知强化学习
arXiv:2606.31825 多模态 评测集 OA · 绿色 被引 1 · S2

在三种多模态 LLM 主干模型上,MRPO 均稳定优于标准 GRPO 及一项最新的 RL 基线;在 Qwen3-VL-8B-Instruct 上甚至超越规模显著更大的医学 MLLM(如 HuatuoGPT-Vision-34B)2.79 分。Across three multimodal LLM backbones, MRPO consistently outperforms standard GRPO and a recent RL baseline, and on Qwen3-VL-8B-Instruct even surpasses substantially larger medical MLLMs such as HuatuoGPT-Vision-34B by 2.79 points.

ABACUS: Adapting Unified Foundation Model for Bridging Image Count Understanding and Generation
ABACUS:适配统一基础模型以桥接图像计数理解与生成
arXiv:2606.23835 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

ABACUS 是一个统一视觉语言模型,可在无需任何基准特定训练的情况下处理物体计数、人群计数、指代表达计数以及忠实计数的图像生成,性能超越任务特定的专家模型和更大的通用模型。ABACUS is a unified vision-language model that handles object counting, crowd counting, referring-expression counting, and count-faithful image generation without any benchmark-specific training required, outperforming both task-specific specialists and larger generalist models.

Video-Oasis: Rethinking Evaluation of Video Understanding
Video-Oasis:重新审视视频理解评估
arXiv:2603.29616 多模态 评测集 OA · 绿色 被引 2 · S2

该审计揭示现有基准样本中 55% 可在无视觉输入或时序上下文的情况下被解决,并提出 Video-Oasis,一个用于系统性审计现有视频理解基准的可持续诊断套件。This audit reveals that 55\% of existing benchmark samples are solvable without visual input or temporal context, and introduces Video-Oasis, a sustainable diagnostic suite for systematically auditing existing video understanding benchmarks.

KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generation
KeyFrame-Compass:迈向关键帧条件视频生成的综合评估
arXiv:2607.14202 多模态 评测集 OA · 绿色 被引 1 · S2

本文将关键帧执行拆解为存在性、保真度、时序顺序、定位、持续性与唯一性六个互补指标,并通过结合专用感知模型的、基于证据的 MLLM 判断来评估整体视频质量。This work decomposes keyframe execution into six complementary metrics covering presence, fidelity, temporal ordering, localization, persistence, and uniqueness, while assessing overall video quality through evidence-grounded MLLM judgments augmented with specialized perception models.

MultiRef-Compass: Towards Comprehensive Evaluation of Multi-Reference-to-Audio-Video Generation
MultiRef-Compass:迈向多参考音视频生成任务的综合评估
arXiv:2607.14189 多模态 评测集 OA · 绿色 被引 1 · S2

本文提出 MultiRef-Compass,一个面向 MR2AV 生成的统一基准,将自动指标与引入复判增强的 MLLM-as-a-Judge 框架相结合,实现对感知保真度与参考条件合成能力的可扩展、可审计评估。MultiRef-Compass is introduced, a unified benchmark for MR2AV generation that integrates automatic metrics with a rejudging-enhanced MLLM-as-a-Judge framework, enabling scalable and auditable evaluation of both perceptual fidelity and reference-conditioned composition.

VIABench: A Comprehensive Video Benchmark Collected from Blind Individuals for Visual Impairment Assistance
VIABench:面向视障辅助任务的盲人视频综合基准
arXiv:2607.14660 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

提出 VIABench,一个专为评估 MLLM 在视障辅助(VIA)场景中表现而设计的综合视频基准,采用视障人士(VIIs)自行录制或分享的第一人称视频,并提出一套严格的评测流水线,同时支持在线(实时)与离线设置。VIABench is introduced, a comprehensive video benchmark specifically designed to evaluate MLLMs in Visually Impaired Assistance scenarios using first-person videos recorded or shared by VIIs themselves, and proposes a rigorous benchmarking pipeline that supports both online (real-time) and offline settings.

ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
ScanNet:富含标注的室内场景三维重建
arXiv:1702.04405 多模态 评测集 OA · 绿色 被引 5793 · S2

本文推出 ScanNet,一个 RGB-D 视频数据集,包含 1513 个场景中的 250 万视角,标注有三维相机位姿、表面重建与语义分割,并表明使用该数据可在多项三维场景理解任务上取得 SOTA 性能。This work introduces ScanNet, an RGB-D video dataset containing 2.5M views in 1513 scenes annotated with 3D camera poses, surface reconstructions, and semantic segmentations, and shows that using this data helps achieve state-of-the-art performance on several 3D scene understanding tasks.

A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
ChatGPT 在推理、幻觉与交互性方面的多任务、多语言、多模态评估
arXiv:2302.04023 多模态 评测集 OA · 绿色 被引 1809 · S2

研究发现 ChatGPT 在大多数任务上以零样本学习优于其他 LLM,在部分任务上甚至超过微调模型,并且对非拉丁文字语言的理解能力优于生成能力。It is found that ChatGPT outperforms LLMs with zero-shot learning on most tasks and even outperforms fine-tuned models on some tasks and is better at understanding non-Latin script languages than generating them.

Matterport3D: Learning from RGB-D Data in Indoor Environments
Matterport3D:基于室内 RGB-D 数据的学习
arXiv:1709.06158 多模态 评测集 OA · 绿色 被引 2631 · S2

本文介绍 Matterport3D,一个大规模 RGB-D 数据集,包含来自 90 个建筑物级场景共 194,400 张 RGB-D 图像的 10,800 个全景视图,可支持多种监督与自监督计算机视觉任务,包括关键点匹配、视角重叠预测、由彩色图像预测法线、语义分割和区域分类。Matterport3D is introduced, a large-scale RGB-D dataset containing 10,800 panoramic views from 194,400RGB-D images of 90 building-scale scenes that enable a variety of supervised and self-supervised computer vision tasks, including keypoint matching, view overlap prediction, normal prediction from color, semantic segmentation, and region classification.

SVR-R1: Bootstrapping Multi-modal Reasoning with Self-verification in Reinforcement Learning
SVR-R1:在强化学习中通过自验证自举多模态推理
arXiv:2607.10966 多模态 评测集 被引 0 · S2

本文提出自验证推理器 SVR-R1,一种多轮强化学习框架,将模型自身的验证转化为多模态推理的学习信号,提供了一种简洁而有效的多模态推理自举方案。Self-Verified Reasoner (SVR-R1), a multi-turn RL framework that turns a model's own verification into a learning signal for multimodal reasoning, is introduced, offering a simple yet effective recipe for bootstrapping multimodal reasoning.

Can Multimodal Large Language Models Understand OCT?
多模态大语言模型能理解OCT吗?
arXiv:2607.16609 多模态 评测集 OA · 绿色 被引 2 · S2

OCT-Bench能够对MLLM进行全面且细粒度的评估,为识别能力瓶颈和推进临床可信的OCT理解奠定基础。OCT-Bench enables comprehensive and fine-grained evaluation of MLLMs, providing a foundation for identifying capability bottlenecks and advancing clinically grounded OCT understanding.

GigaChat Audio: Time-aware Large Audio Language Model
GigaChat Audio:时间感知的大音频语言模型
arXiv:2607.10387 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

本文提出一个时间感知的音频 LLM,能够基于大规模合成监督(来自级联 pipeline)在长达 120 分钟的输入上回答带有显式时间戳的问题,并在短时长和长时长 benchmark 上取得强劲的时间定位准确率。This work presents a time-aware audio LLM that answers questions with explicit timestamps over up to 120 minutes of input using large-scale synthetic supervision from a cascaded pipeline and achieves strong temporal-grounding accuracy on short and long benchmarks.

VQA: Visual Question Answering
VQA: Visual Question Answering
arXiv:1505.00468 多模态 评测集 OA · 绿色 被引 6654 · S2
An Exam for Active Observers
面向主动观察者的评测
arXiv:2607.16165 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

人类视觉是一个闭环:注视点不断被中间假设而非单一快照持续重定向。数十年的心理物理学与认知科学研究表明,主动观察对多种任务至关重要。当代多模态大语言模型 (MLLM) 是否进行主动观察,是一个现有视觉语言基准无法回答的经验问题。我们提出 ActiveVision,一个使 MLLM 主动观察可度量的基准,包含 3 个类别共 17 个任务,任务设计强制进行重复视觉感知……Human vision is a closed loop: gaze is continuously redirected by intermediate hypotheses rather than a single snapshot. Decades of psychophysics and cognitive science have argued that this active observation is essential for a wide range of tasks. Whether today's multimodal large language models (MLLMs) exercise active observation is an empirical question that current vision-language benchmarks do not answer. We introduce ActiveVision, a benchmark that makes active observation measurable for MLLMs, comprising 17 tasks across 3 categories. Tasks are designed to force repeated visual perception

ENTRAP-VL: A Taxonomic Probe for Dual Contextual Entrainment in Vision-Language Models
ENTRAP-VL: A Taxonomic Probe for Dual Contextual Entrainment in Vision-Language Models
arXiv:2607.20092 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

提出 ENTRAP-VL(ENTRainment Assessment Probe for Vision and Language),一个由人工策展的 1,500 条数据的数据集,涵盖八个类别,按一个跨双轴的分类体系组织,并划分为文本诱发流和视觉诱发流。ENTRAP-VL (ENTRainment Assessment Probe for Vision and Language), a manually curated dataset of 1,500 items across eight categories, organized by a taxonomy that spans two axes and split into a textual-entrainment stream and a visual-entrainment stream, is introduced.

Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels
无坐标与区域标签的视觉文档理解中的证据归因
arXiv:2607.24651 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

本研究考察是否存在一条实用路径,在没有坐标界面、且无需高成本区域级监督的条件下提升归因效果,并指出了这样一条可行路径。A study investigates whether there is a practical path to improve attribution without a coordinate interface and without costly region-level supervision, and indicates a practical path to improve attribution without a coordinate interface and without costly region-level supervision.

CLBench-V: Evaluating Multimodal Context Learning from Grounding to Knowledge Acquisition
CLBench-V: 评估多模态上下文学习——从 grounding 到知识获取
arXiv:2607.25294 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

本文介绍 CLBench-V,一个多模态上下文学习 benchmark,围绕三个维度组织任务——上下文 grounding、新信息应用与新知识学习——以解决定位上下文使用失效位置的难题。This work introduces CLBench-V, a benchmark for multimodal context learning that addresses the difficulty of localizing where context use breaks down by organizing tasks around three dimensions: context grounding, new information application, and new knowledge learning.

See2Think: Do Multimodal Models Really Use Intermediate Visual States?
See2Think:多模态模型真的使用了中间视觉状态吗?
arXiv:2607.26769 多模态 评测集 被引 0 · S2

对代表性闭源与开源多模态模型的评测表明,视觉推理强依赖于模型与环境,没有任何单一设置能在所有任务上持续占优。Evaluating representative proprietary and open-source multimodal models, it is found that visual reasoning is strongly model- and environment-dependent, with no single setting consistently dominating across tasks.

AVE-Compass: Towards Holistic Evaluation for Audio-Video Editing Abilities
AVE-Compass:迈向音视频编辑能力的整体评估
arXiv:2607.24821 多模态 评测集 被引 0 · S2

提出 AVE-Agent,一种模块化 agent 框架,将复杂指令分解为相互依赖的子任务,并通过自我反思与评估器反馈迭代改进编辑结果,在联合编辑中提升指令执行、保真度保持以及音视频对齐,同时保持有竞争力的感知质量。AVE-Agent is proposed, a modular agent framework that decomposes complex instructions into dependent subtasks and iteratively improves editing results through self-reflection and evaluator feedback, and improves instruction execution, Fidelity Preserving, and audio-visual alignment in joint editing while maintaining competitive perceptual quality.

SIGNPOST-Bench: Benchmarking Text-Vision Conflict Resolution in Multimodal Large Language Models
SIGNPOST-Bench:面向多模态大语言模型中文本-视觉冲突消解的基准评测
arXiv:2608.04244 多模态 评测集 被引 0 · S2

这些结果将视觉地理定位确立为场景文本仲裁的连续诊断手段,并提供了一个受控框架,用于评估 MLLMs 如何解决冲突的多模态证据。These results establish visual geolocation as a continuous diagnostic of scene-text arbitration and provide a controlled framework for evaluating how MLLMs resolve conflicting multimodal evidence.

Multi-Agent Forensic Reasoning for Generalizable Deepfake Video Detection
多智能体取证推理用于可泛化的深度伪造视频检测
arXiv:2608.06865 多模态 评测集 被引 0 · S2

提出 FaceVid-Forensics-100K,一个大规模深伪视频数据集,包含 100,000 个视频,涵盖 33 种合成方法,覆盖换脸、表情重演与全脸合成;同时提出一个多智能体取证推理框架,由四个领域专家 Agent 分别从四个角度独立分析伪造线索。FaceVid-Forensics-100K is introduced, a large-scale deepfake video dataset comprising 100,000 videos and spanning 33 synthesis methods across face swapping, face reenactment, and entire-face synthesis, and a multi-agent forensic reasoning framework that employs four specialized domain-expert agents to independently analyze forgery cues from four perspectives.

CLIP-CC-Bench: Evaluating Paragraph-Level Video Descriptions in Video-Language Models
CLIP-CC-Bench:评估视频-语言模型中的段落级视频描述
arXiv:2608.04302 多模态 评测集 被引 0 · S2

CLIP-CC-Bench 为长视频描述提供了一个实用的评估框架,填补了现有短片段和仅 QA 基准的空白,并通过评分者间一致性(inter-judge agreement)与 bootstrap 排序稳定性量化该协议的内可靠性。CLIP-CC-Bench provides a practical evaluation framework for long-form video description, filling a gap left by existing short-clip and QA-only benchmarks and quantifying the protocol's internal reliability through inter-judge agreement and bootstrap ranking stability.

H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models
H2R-Bench:世界模型中人到机器人操作视频生成的基准评测
arXiv:2608.13049 多模态 评测集 被引 1 · S2

H2R-Bench 提供了一个系统性诊断框架,用于评估视频世界模型能否跨越 human-to-robot 具身差距,并将人类操作观测转化为以机器人为中心的训练资源。H2R-Bench provides a systematic diagnostic framework for evaluating whether video world models can bridge the human-to-robot embodiment gap and convert human manipulation observations into robot-centric training resources.

A Pathway to General-Purpose Scientific AI: Multimodal Comprehension of Scientific Images
迈向通用科学 AI 的路径:科学图像的多模态理解
arXiv:2608.14075 多模态 评测集 被引 1 · S2

展望 ALD/E-ImageMiner benchmark 如何指导未来科学图像挑战赛,以及 Bloom-informed 问题设计如何支持更深入的科学理解。A forward-looking perspective is presented on how the ALD/E-ImageMiner benchmark can guide future scientific-image challenges and how Bloom-informed question design can support deeper scientific understanding.

TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation
TRACE-Bench:多参考图像生成的分解与诊断
arXiv:2608.16765 多模态 评测集 被引 0 · S2

认识到多样化的多参考任务共享一组共同的原子操作,本文形式化了四个算子:Anchor、Disentangle、Apply 和 Compose,并构建了 TRACE-Bench,包含约 1,600 个跨 slot 数量 1–8 的评估用例。Recognizing that diverse multi-reference tasks share a common set of atomic operations, this work formalizes four operators: Anchor, Disentangle, Disentangle, Apply, and Compose, and constructs TRACE-Bench, comprising approximately 1,600 evaluation cases across slot counts 1--8.