Papers · organized/paper_cards

论文

20 张论文卡片 · 多模态 · 应用落地

开放获取 全部 绿色 · 677
DrivePI: Spatial-aware 4D MLLM for Unified Autonomous Driving
DrivePI:面向统一自动驾驶的空间感知 4D MLLM
arXiv:/inbox/flyp/2026-06-11-DrivePI-4D-MLLM-autonomous-driving.md 多模态 应用落地
Embodied-Navigator: Point, Think, Memorize, and Align for Efficient Navigation
Embodied-Navigator:用于高效导航的指向、思考、记忆与对齐
arXiv:2608.17512 多模态 应用落地 被引 0 · S2

该工作提出 TAMP-Nav,一个用于高效具身导航的统一框架,可在关键节点动态触发 Chain-of-Thought 并仅在关键节点保留高保真记忆,将冗余轨迹压缩为轻量级时空指示器,从而保留关键历史信息并增强时空感知。This work proposes TAMP-Nav, a unified framework for efficient embodied navigation that dynamically triggers Chain-of-Thought and retains high-fidelity memory only at critical nodes, compressing redundant trajectories into lightweight Space-Time Indicators, thereby preserving critical historical information and enhancing spatio-temporal perception.

From Foundation to Application: Improving VLA Models in Practice
从基础到应用:实践中改进 VLA 模型
arXiv:2607.06403 多模态 应用落地 OA · 绿色 被引 11 · S2

得益于涵盖全身自由度的扩展预训练数据,LingBot-VLA-2.0 在两个机器人平台上展现出强大的跨具身长时程移动操作能力。Benefiting from the expanded pretraining data that covers whole-body degrees of freedom, LingBot-VLA-2.0 demonstrates strong cross-embodiment long-horizon mobile manipulation capability across the two robotic platforms.

MultAttnAttrib: Training-Free Multimodal Attribution in Long Document Question Answering
MultAttnAttrib:长文档问答中的免训练多模态归因
arXiv:2607.01420 多模态 应用落地 OA · 绿色 被引 0 · S2 + OpenAlex

本工作提出 MultAttnAttrib,一种免训练的归因生成方法,利用模型的预填充过程、选定的注意力头以及校准阈值在文档中定位源证据,且在多种归因生成方法上一致地表现更优。This work introduces MultAttnAttrib, a training-free attribution-generation method that leverages a model's prefill pass, selected attention heads, and calibrated thresholds to locate source evidence within a document, and consistently outperforms a variety of attribution-generation methods.

JD Oxygen AI Item Center (Oxygen AIIC) V1: An Industrial-Scale LLM/VLM-Centric Solution for Item Understanding, Management, and Applications
京东 Oxygen AI 商品中心(Oxygen AIIC)V1:以 LLM/VLM 为核心的工业级商品理解、管理与应用解决方案
arXiv:2606.28070 多模态 应用落地 OA · 绿色 被引 0 · S2 + OpenAlex

京东 Oxygen AI 商品中心(Oxygen AIIC):基于 LLM/VLM 的工业级商品知识生产与服务平台,已在大规模场景下取得可量化的收益。The JD Oxygen AI Item Center (Oxygen AIIC), an industrial-scale platform built on LLMs/VLMs for item-knowledge production and service, has delivered measurable gains at scale.

Vesta: A Generalist Embodied Reasoning Model
Vesta:通用具身推理模型
arXiv:2606.20905 多模态 应用落地 OA · 绿色 被引 2 · S2

提出 Vesta,一个统一的具身通用模型,将定位、空间推理、导航和长程规划能力整合到单个基础模型中,并证明通用模型能够达到或超越专家模型。Vesta is presented, a unified embodied generalist that consolidates localization, spatial reasoning, navigation, navigation, and long-horizon planning capabilities into a single foundation model and demonstrates that a generalist model can match or exceed specialists.

Object-Centric Residual RL for Zero-Shot Sim-to-Real VLA Enhancement
基于物体中心的残差强化学习用于 VLA 零样本仿真到现实迁移的增强
arXiv:2606.18953 多模态 应用落地 OA · 绿色 被引 0 · S2 + OpenAlex

提出一种基于物体中心的残差强化学习框架,利用物体位姿精化 VLA 动作,使观测空间紧凑,在仿真与现实之间能够一致迁移。An object-centric residual RL framework is proposed that refines VLA actions using object poses, enabling a compact observation space that transfers consistently between simulation and reality.

A Quantized Native Runtime for On-Device Semantic Audio Generation
一种用于端侧语义音频生成的量化原生运行时
arXiv:2607.08526 多模态 应用落地 OA · 绿色 被引 0 · S2 + OpenAlex

该紧凑且量化原生、带内置控制的运行时为物联网音频场景下的端侧语义音频提供了实用基础;通过对转向接口的案例分析,可生成在部分属性上具有真实但有界控制的、承载口味联想的音乐。A compact, quantized runtime with built-in control a practical basis for on-device semantic audio in Internet-of-Sounds settings and a case study of the steering interface generates music carrying taste associations with genuine but bounded control for a subset of attributes.

Motion4Motion: Motion Transfer Across Subjects at Inference
Motion4Motion:推理阶段的跨主体运动迁移
arXiv:2607.11644 多模态 应用落地 OA · 绿色 被引 0 · S2 + OpenAlex

Motion4Motion 对视频中角色的 motion flow(而非骨骼)进行建模,使跨物种运动迁移更加容易。Motion4Motionmodels the motion flow of the character in a video instead of skeletons, which makes motion transfer across species easier, which makes motion transfer across species easier.

Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model
Xiaomi-Robotics-U0:基于 World Foundation Model 的统一具身合成
arXiv:2607.11643 多模态 应用落地 OA · 绿色 被引 1 · S2

Xiaomi-Robotics-U0 是首个支持跨多种机器人本体的高质量多视角场景生成、并引入结构化、可控的具身迁移以实现细粒度编辑的模型,同时保持多视角一致性与交互动态。Xiaomi-Robotics-U0 is the first model to support high-quality multi-view scene generation across multiple robot embodiments and to introduce structured, controllable embodied transfer for fine-grained editing while preserving multi-view consistency and interaction dynamics.

VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding
VideoChat3:面向高效通用视频理解的完全开放视频 MLLM
arXiv:2607.14935 多模态 应用落地 OA · 绿色 被引 4 · S2

本文提出 VideoChat3,一个完全开放、高效且以视频为中心的通用 MLLM,仅以 4B 参数与更高效率,超越参数量相当或更大的已有开源模型。This work introduces VideoChat3, a fully open, efficient, and generalist video-centric MLLM, which surpasses prior open-source models with equal or larger parameter counts with only 4B parameters and higher efficiency.

Robostral Navigate
Robostral Navigate
arXiv:2607.20785 多模态 应用落地 OA · 绿色 被引 0 · S2 + OpenAlex

提出 Robostral Navigate,一个围绕该可扩展性目标构建的 8B 视觉语言模型,仅消费单目 RGB 图像流——这是机器人平台中最普及的传感器——通过在当前相机画面中指向下一目标位置来预测航点。Robostral Navigate, an 8B vision-language model built around this scalability objective, is introduced, which consumes only a stream of monocular RGB images - the most ubiquitous sensor across robotic platforms and predicts waypoints by pointing to the next target location in the current camera view.

Closing the Loop: Training-Free Revisit Consistency for Autoregressive Generative Rendering
Closing the Loop:面向自回归生成式渲染的无训练 Revisit 一致性
arXiv:2607.21848 多模态 应用落地

近期条件视频生成模型已展现出将 3D 引擎渲染(如深度图与无纹理几何体)转化为照片级真实视频的潜力,可应用于游戏与沉浸式内容创作。此类应用要求长时程自回归生成,在持续合成新帧的同时维持持久的 3D 世界。自回归生成器以有界 KV cache 逐 chunk 合成视频,因此当相机再次访问已从上下文中驱逐的位置时,模型常会重新生成不一致的外观,尽管该Recent conditional video generation models have shown promising potentials to transform 3D engine renderings, such as depth maps and untextured geometry, into photorealistic videos for gaming and immersive content creation. These applications require long-horizon auto-regressive generation that continuously synthesizes new frames while preserving a persistent 3D world. Auto-regressive generators synthesize video chunk by chunk with a bounded KV cache, so when the camera revisits a location after its context has been evicted, the model often regenerates inconsistent appearance, even though the

UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models
UltraViT:面向大视觉-语言模型的端侧延迟优化视觉编码器
arXiv:2607.23373 多模态 应用落地 OA · 绿色 被引 0 · S2 + OpenAlex

大量实验表明,结合端上延迟感知设计与定制化训练策略,建立了高效 LVLM 编码的新 SOTA,在端上以近 1.7 倍速度运行的同时显著优于现有以编码器为中心的基线。Extensive experiments demonstrate that the on-device latency-informed design combined with the tailored training strategy establishes a new state-of-the-art for efficient LVLM encoding, significantly outperforming existing encoder-centric baselines while operating on-device at nearly 1.7xthe speed.

Human-in-the-Loop Signature Bootstrapping for UAV Hyperspectral PFM-1 Mine Detection
UAV 高光谱 PFM-1 地雷检测中的人在回路签名引导
arXiv:2607.25310 多模态 应用落地 OA · 绿色 被引 1 · S2

本文研究无人机(UAV)可见光-近红外(VNIR)高光谱图像中 PFM-1 地雷的检测,使用光谱角制图(SAM)、匹配滤波器(MF)、自适应相干估计器(ACE)和约束能量最小化(CEM)。This paper studies PFM-1 landmine detection in unmanned aerial vehicle (UAV) visible and near-infrared (VNIR) HSI using spectral angle mapper (SAM), matched filter (MF), adaptive coherence estimator (ACE), and constrained energy minimization (CEM).

Loud or Silent? A Reusable Framework for Per-Modality Failure Analysis in Multimodal Clinical AI
标题 -> 标题中文:响亮还是沉默?面向多模态临床 AI 的可复用逐模态失效分析框架
arXiv:2608.01462 多模态 应用落地 被引 0 · S2

一种模型无关的模态失败框架,仅依赖部署可观察信号,返回逐样本失败分类、逐模态互补矩阵(将错误归因到模态)以及响亮 vs 静默 dropout 画像(区分可监控失败与远离决策边界未被标记的失败)A model-agnostic modality-failure framework that returns a per-example failure taxonomy, a per-modality complementarity matrix that attributes error to modalities, and a loud-vs-silent dropout profile separating monitorable failures from those that pass unflagged far from the decision boundary, using only deployment-observable signals.

BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulation
BridgeVLA++:用于 3D 操作的、数据高效、可泛化且具备记忆增强的 Vision-Language-Action 框架
arXiv:2608.05042 多模态 应用落地 被引 0 · S2

在 BridgeVLA 基础上开发 BridgeVLA++,引入统一的时空记忆架构,建模持久化的空间上下文与时间交互历史,使其可在保留 BridgeVLA 数据效率与泛化能力的同时对观测历史进行推理。BridgeVLA++ is developed by equipping BridgeVLA with a unified spatio-temporal memory architecture that models persistent spatial context and temporal interaction history that can reason over observation histories while preserving BridgeVLA's data efficiency and generalization capabilities.

Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors
往返一致性:双向扩散模型可预测自身的 rollout 误差
arXiv:2608.00675 多模态 应用落地 被引 1 · S2

往返一致性将可逆性转化为生成式模型一种实用的可信信号;双向训练带来负成本,在两个方向上均优于单向专家模型;其中反向还可作为快速的逆问题求解器。Round-trip consistency turns reversibility into a practical trust signal for generative models, and Bidirectional training comes at negative cost, beating direction specialists in both directions, and the backward direction doubles as a fast inverse solver.

Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networks
使用正则化深度 LSTM 网络进行基于骨骼动作识别的共现特征学习
arXiv:1603.07772 多模态 应用落地 OA · 绿色 被引 930 · S2

本文在每个时间步以骨骼作为输入,引入一种新的正则化方案来学习骨骼关节的共现特征,并提出一种同时作用于 LSTM 神经元门、单元和输出响应的新型 dropout 算法。This work takes the skeleton as the input at each time slot and introduces a novel regularization scheme to learn the co-occurrence features of skeleton joints, and proposes a new dropout algorithm which simultaneously operates on the gates, cells, and output responses of the LSTM neurons.

LiveAnimate: Stable Long-Form Streaming Human Animation in Real-Time
LiveAnimate:实时稳定的长视频流式人体动画
arXiv:2608.11745 多模态 应用落地 被引 0 · S2

本文提出 LiveAnimate,据作者所知是首个将实时流式生成与十亿参数规模下的稳定长视频生成相结合的系统,基于 140 亿参数的视频 Diffusion Transformer(DiT)。This work presents LiveAnimate, to their knowledge the first animation system to combine real-time streaming with stable long-form generation at billion scale, built on a 14B-parameter video Diffusion Transformer (DiT).