研究发现,recall@k 并非已部署 KB-VQA 的正确评价指标,且弥合差距需要 reader 侧介入;本文首次对多模态 KB-VQA 中 reader 侧位置依赖性进行了受控探查,设计了一种 gold-position 协议——在问题提示中仅改变 gold passage 所在的槽位。The findings indicate that recall@k is the wrong metric for deployed KB-VQA and that closing the gap requires reader-side intervention; the first controlled probe of reader-side position dependence in multimodal KB-VQA is designed, a gold-position protocol in which only the gold passage's prompt slot varies within question.
论文
15 张论文卡片 · 多模态
本文提出 Vision-aligned Latent Reasoning(VaLR),一种简洁而有效的推理框架,在每个 Chain of Thought 推理步骤之前动态生成视觉对齐的 latent token,引导模型在 latent space 中基于感知线索进行推理。Vision-aligned Latent Reasoning (VaLR) is introduced, a simple, yet effective reasoning framework that dynamically generates vision-aligned latent tokens before each Chain of Thought reasoning step, guiding the model to reason based on perceptual cues in the latent space.
本文提出 ContextRL,一种上下文感知的强化学习方法,通过间接辅助目标提升长周期推理与多模态性能,并与将相同对比上下文复用作标准 query–context–answer 样本的数据增强基线进行对比。This work proposes ContextRL, a context-aware reinforcement learning (RL) method that improves long-horizon reasoning and multimodal performance through an indirect auxiliary objective, and compares against data-augmentation baselines that repurpose the same contrastive contexts as standard query--context--answer examples.
本研究建立了一套实用的 LongPT 方案,为推进长上下文 vision-language 模型奠定了经验基础,并提出 MMProLong,无需任务专属监督即可泛化至基于网页的多模态 needle 检索、长上下文图文压缩以及长视频理解等任务。This study establishes a practical LongPT recipe and an empirical foundation for advancing long-context vision-language models, and introduces MMProLong, which generalizes to webpage-based multimodal needle retrieval, long-context vision-text compression, and long-video understanding without task-specific supervision.
本文提出 LLaDA-V,一种完全基于扩散范式的多模态大语言模型 (MLLM),将视觉指令微调与 masked diffusion 模型相结合,脱离了当前多模态方法中主流的自回归范式。LLaDA-V is introduced, a purely diffusion-based Multimodal Large Language Model (MLLM) that integrates visual instruction tuning with masked diffusion models, representing a departure from the autoregressive paradigms dominant in current multimodal approaches.
本文概述了第二届 MAGMaR(Multimodal Retrieval 驱动的多模态增强生成)研讨会共享任务的成果,参赛系统聚焦于视频检索,或在给定检索视频的基础上进行有依据的文章生成。This overview paper presents the results of the shared task for the second workshop on Multimodal Augmented Generation via Multimodal Retrieval (MAGMaR), where participants submitted systems focused on either video retrieval or grounded generation of articles given retrieved videos.
ViT-5 的设计与当代基础模型实践保持一致,可作为对 vanilla ViT 的直接替换升级方案,适用于 2020 年代中期的视觉骨干网络,并为生成建模提供更强大的骨干。With a design aligned with contemporary foundation-model practices, ViT-5 offers a simple drop-in upgrade over vanilla ViT for mid-2020s vision backbones and serves as a stronger backbone for generative modeling.
构建了 RoboGenesis,一个基于仿真的工作流与数据引擎,可从原子技能组合配置好的实验工作流,对 rollout 进行验证与过滤,并跨支持的机器人配置导出结构化演示数据。RoboGenesis is built, a simulation-based workflow and data engine that composes configured laboratory workflows from atomic skills, validates and filters rollouts, and exports structured demonstrations across supported robot profiles.
本文提出 DashboardQA,这是首个明确设计用于评估视觉-语言 GUI Agent 对真实世界仪表板理解与交互能力的基准,结果表明交互式仪表板推理对所有受评估的 VLM 而言都是一项具有挑战性的任务。DashboardQA is introduced, the first benchmark explicitly designed to assess how vision-language GUI agents comprehend and interact with real-world dashboards, and indicates that interactive dashboard reasoning is a challenging task overall for all the VLMs evaluated.
本文基于预训练 BLIP-2 模型,对视觉-语言指令微调展开系统全面研究,并提出指令感知的 Query Transformer,用于提取针对给定指令的信息丰富特征。This paper conducts a systematic and comprehensive study on vision-language instruction tuning based on the pretrained BLIP-2 models, and introduces an instruction-aware Query Transformer, which extracts informative features tailored to the given instruction.
深入回顾文献中 GAN 相关研究,并从两个视角阐述针对三大挑战所提出的架构变体与损失变体。An in-depth review of GAN-related research in the literature is provided, and an account of the architecture-variant and loss-variants, which have been proposed to handle these three challenges from two perspectives are provided.
综述领域自适应与迁移学习,重点关注视觉应用及超越图像分类的方法,如目标检测、图像分割、视频分析或视觉属性学习。An overview of domain adaptation and transfer learning with a specific view on visual applications and the methods that go beyond image categorization, such as object detection or image segmentation, video analyses or learning visual attributes are overviewed.