Papers · organized/paper_cards

论文

19 张论文卡片 · 多模态 · 评测集 · OA 绿色

开放获取 全部 绿色 · 724
DataComp-VLM: Improved Open Datasets for Vision-Language Models
DataComp-VLM:面向视觉-语言模型的改进开源数据集
arXiv:2606.28551 多模态 评测集 MPG.PuRe (Max Planck Society) OA · 绿色 被引 1 · S2

数据混合(而非过滤)是构建高质量训练数据集的关键:以指令型数据为主的混合在扩展时优于以描述型数据为主的混合,且规模越大优势越明显。It is found that data mixing, not filtering, is key to a high-quality training dataset: instruction-heavy mixtures scale better than caption-heavy ones, with gains widening at larger scales.

Discrete Diffusion Language Models for Interactive Radiology Report Drafting
用于交互式放射学报告起草的离散扩散语言模型
arXiv:2607.01436 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

本文适配了一款专家混合扩散语言模型 DiffusionGemma-26B,并在医学视觉问答数据集上,使用相同的 LoRA 配置将其与同规模的自回归模型 Gemma-4-26B 进行基准对比,由对冗长度鲁棒的 LLM 裁判打分。This work adapts a mixture-of-experts diffusion language model, DiffusionGemma-26B, and benchmark it against its same-size AR sibling Gemma-4-26B under an identical LoRA recipe on medical visual question answering datasets, scored by a verbosity-robust LLM judge.

Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning
打破失败级联:面向医学多模态推理的步骤感知强化学习
arXiv:2606.31825 多模态 评测集 OA · 绿色 被引 1 · S2

在三种多模态 LLM 主干模型上,MRPO 均稳定优于标准 GRPO 及一项最新的 RL 基线;在 Qwen3-VL-8B-Instruct 上甚至超越规模显著更大的医学 MLLM(如 HuatuoGPT-Vision-34B)2.79 分。Across three multimodal LLM backbones, MRPO consistently outperforms standard GRPO and a recent RL baseline, and on Qwen3-VL-8B-Instruct even surpasses substantially larger medical MLLMs such as HuatuoGPT-Vision-34B by 2.79 points.

ABACUS: Adapting Unified Foundation Model for Bridging Image Count Understanding and Generation
ABACUS:适配统一基础模型以桥接图像计数理解与生成
arXiv:2606.23835 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

ABACUS 是一个统一视觉语言模型,可在无需任何基准特定训练的情况下处理物体计数、人群计数、指代表达计数以及忠实计数的图像生成,性能超越任务特定的专家模型和更大的通用模型。ABACUS is a unified vision-language model that handles object counting, crowd counting, referring-expression counting, and count-faithful image generation without any benchmark-specific training required, outperforming both task-specific specialists and larger generalist models.

Video-Oasis: Rethinking Evaluation of Video Understanding
Video-Oasis:重新审视视频理解评估
arXiv:2603.29616 多模态 评测集 OA · 绿色 被引 2 · S2

该审计揭示现有基准样本中 55% 可在无视觉输入或时序上下文的情况下被解决,并提出 Video-Oasis,一个用于系统性审计现有视频理解基准的可持续诊断套件。This audit reveals that 55\% of existing benchmark samples are solvable without visual input or temporal context, and introduces Video-Oasis, a sustainable diagnostic suite for systematically auditing existing video understanding benchmarks.

KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generation
KeyFrame-Compass:迈向关键帧条件视频生成的综合评估
arXiv:2607.14202 多模态 评测集 OA · 绿色 被引 1 · S2

本文将关键帧执行拆解为存在性、保真度、时序顺序、定位、持续性与唯一性六个互补指标,并通过结合专用感知模型的、基于证据的 MLLM 判断来评估整体视频质量。This work decomposes keyframe execution into six complementary metrics covering presence, fidelity, temporal ordering, localization, persistence, and uniqueness, while assessing overall video quality through evidence-grounded MLLM judgments augmented with specialized perception models.

MultiRef-Compass: Towards Comprehensive Evaluation of Multi-Reference-to-Audio-Video Generation
MultiRef-Compass:迈向多参考音视频生成任务的综合评估
arXiv:2607.14189 多模态 评测集 OA · 绿色 被引 1 · S2

本文提出 MultiRef-Compass,一个面向 MR2AV 生成的统一基准,将自动指标与引入复判增强的 MLLM-as-a-Judge 框架相结合,实现对感知保真度与参考条件合成能力的可扩展、可审计评估。MultiRef-Compass is introduced, a unified benchmark for MR2AV generation that integrates automatic metrics with a rejudging-enhanced MLLM-as-a-Judge framework, enabling scalable and auditable evaluation of both perceptual fidelity and reference-conditioned composition.

VIABench: A Comprehensive Video Benchmark Collected from Blind Individuals for Visual Impairment Assistance
VIABench:面向视障辅助任务的盲人视频综合基准
arXiv:2607.14660 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

提出 VIABench,一个专为评估 MLLM 在视障辅助(VIA)场景中表现而设计的综合视频基准,采用视障人士(VIIs)自行录制或分享的第一人称视频,并提出一套严格的评测流水线,同时支持在线(实时)与离线设置。VIABench is introduced, a comprehensive video benchmark specifically designed to evaluate MLLMs in Visually Impaired Assistance scenarios using first-person videos recorded or shared by VIIs themselves, and proposes a rigorous benchmarking pipeline that supports both online (real-time) and offline settings.

ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
ScanNet:富含标注的室内场景三维重建
arXiv:1702.04405 多模态 评测集 OA · 绿色 被引 5793 · S2

本文推出 ScanNet,一个 RGB-D 视频数据集,包含 1513 个场景中的 250 万视角,标注有三维相机位姿、表面重建与语义分割,并表明使用该数据可在多项三维场景理解任务上取得 SOTA 性能。This work introduces ScanNet, an RGB-D video dataset containing 2.5M views in 1513 scenes annotated with 3D camera poses, surface reconstructions, and semantic segmentations, and shows that using this data helps achieve state-of-the-art performance on several 3D scene understanding tasks.

A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
ChatGPT 在推理、幻觉与交互性方面的多任务、多语言、多模态评估
arXiv:2302.04023 多模态 评测集 OA · 绿色 被引 1809 · S2

研究发现 ChatGPT 在大多数任务上以零样本学习优于其他 LLM,在部分任务上甚至超过微调模型,并且对非拉丁文字语言的理解能力优于生成能力。It is found that ChatGPT outperforms LLMs with zero-shot learning on most tasks and even outperforms fine-tuned models on some tasks and is better at understanding non-Latin script languages than generating them.

Matterport3D: Learning from RGB-D Data in Indoor Environments
Matterport3D:基于室内 RGB-D 数据的学习
arXiv:1709.06158 多模态 评测集 OA · 绿色 被引 2631 · S2

本文介绍 Matterport3D,一个大规模 RGB-D 数据集,包含来自 90 个建筑物级场景共 194,400 张 RGB-D 图像的 10,800 个全景视图,可支持多种监督与自监督计算机视觉任务,包括关键点匹配、视角重叠预测、由彩色图像预测法线、语义分割和区域分类。Matterport3D is introduced, a large-scale RGB-D dataset containing 10,800 panoramic views from 194,400RGB-D images of 90 building-scale scenes that enable a variety of supervised and self-supervised computer vision tasks, including keypoint matching, view overlap prediction, normal prediction from color, semantic segmentation, and region classification.

Can Multimodal Large Language Models Understand OCT?
多模态大语言模型能理解OCT吗?
arXiv:2607.16609 多模态 评测集 OA · 绿色 被引 2 · S2

OCT-Bench能够对MLLM进行全面且细粒度的评估,为识别能力瓶颈和推进临床可信的OCT理解奠定基础。OCT-Bench enables comprehensive and fine-grained evaluation of MLLMs, providing a foundation for identifying capability bottlenecks and advancing clinically grounded OCT understanding.

GigaChat Audio: Time-aware Large Audio Language Model
GigaChat Audio:时间感知的大音频语言模型
arXiv:2607.10387 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

本文提出一个时间感知的音频 LLM,能够基于大规模合成监督(来自级联 pipeline)在长达 120 分钟的输入上回答带有显式时间戳的问题,并在短时长和长时长 benchmark 上取得强劲的时间定位准确率。This work presents a time-aware audio LLM that answers questions with explicit timestamps over up to 120 minutes of input using large-scale synthetic supervision from a cascaded pipeline and achieves strong temporal-grounding accuracy on short and long benchmarks.

VQA: Visual Question Answering
VQA: Visual Question Answering
arXiv:1505.00468 多模态 评测集 OA · 绿色 被引 6654 · S2
An Exam for Active Observers
面向主动观察者的评测
arXiv:2607.16165 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

人类视觉是一个闭环:注视点不断被中间假设而非单一快照持续重定向。数十年的心理物理学与认知科学研究表明,主动观察对多种任务至关重要。当代多模态大语言模型 (MLLM) 是否进行主动观察,是一个现有视觉语言基准无法回答的经验问题。我们提出 ActiveVision,一个使 MLLM 主动观察可度量的基准,包含 3 个类别共 17 个任务,任务设计强制进行重复视觉感知……Human vision is a closed loop: gaze is continuously redirected by intermediate hypotheses rather than a single snapshot. Decades of psychophysics and cognitive science have argued that this active observation is essential for a wide range of tasks. Whether today's multimodal large language models (MLLMs) exercise active observation is an empirical question that current vision-language benchmarks do not answer. We introduce ActiveVision, a benchmark that makes active observation measurable for MLLMs, comprising 17 tasks across 3 categories. Tasks are designed to force repeated visual perception

ENTRAP-VL: A Taxonomic Probe for Dual Contextual Entrainment in Vision-Language Models
ENTRAP-VL: A Taxonomic Probe for Dual Contextual Entrainment in Vision-Language Models
arXiv:2607.20092 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

提出 ENTRAP-VL(ENTRainment Assessment Probe for Vision and Language),一个由人工策展的 1,500 条数据的数据集,涵盖八个类别,按一个跨双轴的分类体系组织,并划分为文本诱发流和视觉诱发流。ENTRAP-VL (ENTRainment Assessment Probe for Vision and Language), a manually curated dataset of 1,500 items across eight categories, organized by a taxonomy that spans two axes and split into a textual-entrainment stream and a visual-entrainment stream, is introduced.

Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels
无坐标与区域标签的视觉文档理解中的证据归因
arXiv:2607.24651 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

本研究考察是否存在一条实用路径,在没有坐标界面、且无需高成本区域级监督的条件下提升归因效果,并指出了这样一条可行路径。A study investigates whether there is a practical path to improve attribution without a coordinate interface and without costly region-level supervision, and indicates a practical path to improve attribution without a coordinate interface and without costly region-level supervision.

CLBench-V: Evaluating Multimodal Context Learning from Grounding to Knowledge Acquisition
CLBench-V: 评估多模态上下文学习——从 grounding 到知识获取
arXiv:2607.25294 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

本文介绍 CLBench-V,一个多模态上下文学习 benchmark,围绕三个维度组织任务——上下文 grounding、新信息应用与新知识学习——以解决定位上下文使用失效位置的难题。This work introduces CLBench-V, a benchmark for multimodal context learning that addresses the difficulty of localizing where context use breaks down by organizing tasks around three dimensions: context grounding, new information application, and new knowledge learning.

See2Think: Do Multimodal Models Really Use Intermediate Visual States?
See2Think:多模态模型真的使用了中间视觉状态吗?
arXiv:2607.26769 多模态 评测集 OA · 绿色 被引 0 · S2 + OpenAlex

对代表性闭源与开源多模态模型的评测表明,视觉推理强依赖于模型与环境,没有任何单一设置能在所有任务上持续占优。Evaluating representative proprietary and open-source multimodal models, it is found that visual reasoning is strongly model- and environment-dependent, with no single setting consistently dominating across tasks.