Papers · organized/paper_cards

论文

162 张论文卡片 · 多模态 · OA 绿色

开放获取 全部 绿色 · 724
Not All Tokens Deserve Equal Credit: Counterfactual Sensitivity Credit Reallocation for Long-CoT Reasoning
并非所有 token 都应获得同等贡献:面向长链思维推理的反事实敏感度贡献重分配
arXiv:2607.27888 多模态 方法 OA · 绿色 被引 0 · S2 + OpenAlex

研究发现privileged shifts无法给出可靠的答案对齐方向,其幅度主要反映反事实敏感性而非token级学习价值;提出Counterfactual Sensitivity Credit Reallocation (CSCR),作为GRPO的简单扩展,降低高敏感token的credit并对token级优势重新归一化,同时保留原始credit预算与verifier确定的方向。These findings show that privileged shifts fail to provide reliable answer-aligned directions, while their magnitudes primarily reflect counterfactual sensitivity rather than token-level learning value, and propose Counterfactual Sensitivity Credit Reallocation (CSCR), a simple extension of GRPO that reduces credit for highly sensitive tokens and renormalizes token-level advantages to preserve both the original credit budget and verifier-determined direction.

Motion Beyond Morphology: Bootstrapping Cross-Category Motion Transfer from Abstract Motion Representations
超越形态的运动:从抽象运动表示引导跨类别运动迁移
arXiv:2608.01628 多模态 方法 OA · 绿色 被引 0 · S2 + OpenAlex

本文提出 Motion Beyond Morphology(超越形态的运动)这一视角,旨在跨固定结构对应迁移运动,通过两阶段框架保留在不同目标形态间仍具有意义的动力学。This work introduces Motion Beyond Morphology, a perspective that seeks to transfer motion beyond fixed structural correspondence, by preserving dynamics that remain meaningful across different target morphologies by proposing a two-stage framework.

3DZip: Spatial-Aware Feature Diversity-Guided Token Compression for 3D Question Answering
3DZip:面向 3D 问答的空间感知特征多样性引导 token 压缩
arXiv:2608.01185 多模态 方法 OA · 绿色 被引 0 · S2 + OpenAlex

本文提出 3DZip,一种三阶段 token 压缩框架:首先采用粗粒度体素化去除点级冗余,再通过 Determinantal Point Process 基于特征空间多样性选取锚点 token,最后在空间约束下融合剩余 token 以保持几何一致性。3DZip is proposed, a three-stage token compression framework that first applies coarse voxelization to remove point-level redundancy, then selects anchor tokens based on feature-space diversity via a Determinantal Point Process, and finally merges remaining tokens under spatial constraints to preserve geometric coherence.

LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation
LeapTalk:打破 Talking Head 生成中的延迟-质量权衡
arXiv:2608.00079 多模态 方法 OA · 绿色 被引 0 · S2 + OpenAlex

本文提出 LeapTalk,一种通过单次前向实现稳定且实时说话头生成、可扩展至任意长视频的新颖框架,并引入音频驱动的无分类器引导机制,在极端步数缩减下保持细粒度唇形同步。LeapTalk is proposed, a novel framework that achieves stable and real-time talking-head generation with a single forward step, scaling to arbitrarily long videos, and an audio-driven classifier-free guidance mechanism to maintain fine-grained lip synchronization under extreme step reduction.

Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
用于视觉问答与视觉定位的多模态紧凑双线性池化
arXiv:1606.01847 多模态 方法 OA · 绿色 被引 1602 · S2

在视觉问答与视觉定位任务上对多模态紧凑双线性池化(MCB)进行了广泛评测,结果一致表明 MCB 优于去掉 MCB 的消融版本This work extensively evaluates Multimodal Compact Bilinear pooling (MCB) on the visual question answering and grounding tasks and consistently shows the benefit of MCB over ablations without MCB.

TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
TransUNet: Transformers 作为医学图像分割的强大编码器
arXiv:2102.04306 多模态 方法 OA · 绿色 被引 6199 · S2

文章论证了 Transformers 可作为医学图像分割任务的强大编码器,并通过与 U-Net 结合,恢复了局部空间信息以增强更精细的细节。It is argued that Transformers can serve as strong encoders for medical image segmentation tasks, with the combination of U-Net to enhance finer details by recovering localized spatial information.

SegFormer: Simple and Efficient Design for Semantic Segmentation with\n Transformers
SegFormer: 基于 Transformers 的简单高效语义分割设计
arXiv:2105.15203 多模态 方法 OA · 绿色 被引 9208 · S2

文章提出了 SegFormer,一个简单高效且强大的语义分割框架,将 Transformers 与轻量级 MLP 解码器统一,并在 Cityscapes-C 上展示了出色的零样本鲁棒性。SegFormer is presented, a simple, efficient yet powerful semantic segmentation framework which unifies Transformers with lightweight multilayer perception (MLP) decoders and shows excellent zero-shot robustness on Cityscapes-C.

VisualBERT: A Simple and Performant Baseline for Vision and Language
VisualBERT: 一个简单且性能优异的视觉与语言基线模型
arXiv:1908.03557 多模态 方法 OA · 绿色 被引 2375 · S2

分析表明,VisualBERT 能够在无任何显式监督的情况下将语言元素对应到图像区域,甚至对句法关系敏感,例如能够跟踪动词与其论元对应图像区域之间的关联。Analysis demonstrates that VisualBERT can ground elements of language to image regions without any explicit supervision and is even sensitive to syntactic relationships, tracking, for example, associations between verbs and image regions corresponding to their arguments.

Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation
Swin-Unet: 类 U-Net 的纯 Transformer 医学图像分割网络
arXiv:2105.05537 多模态 方法 OA · 绿色 被引 5572 · S2

在输入和输出直接进行 4 倍下采样与上采样的设定下,实验表明,基于纯 Transformer 的 U 形编码器-解码器网络优于完全卷积或 Transformer 与卷积相结合的方法。Under the direct down-sampling and up-sampled of the inputs and outputs by 4x, experiments demonstrate that the pure Transformer-based U-shaped Encoder-Decoder network outperforms those methods with full Convolution or the combination of transformer and convolution.

BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
BLIP: 面向统一视觉-语言理解与生成的引导式语言-图像预训练
arXiv:2201.12086 多模态 方法 OA · 绿色 被引 7222 · S2

BLIP 通过引导式 caption 方式有效利用含噪网络数据,由 captioner 生成合成 caption,并由 filter 去除噪声样本;在以零样本方式直接迁移到视频-语言任务时,展现出强大的泛化能力。BLIP effectively utilizes the noisy web data by bootstrapping the captions, where a captioner generates synthetic captions and a filter removes the noisy ones, and demonstrates strong generalization ability when directly transferred to video-language tasks in a zero-shot manner.

A deep learning architecture for temporal sleep stage classification\n using multivariate and multimodal time series
基于多变量多模态时间序列的深度学习睡眠阶段时序分类架构
arXiv:1707.03321 多模态 方法 OA · 绿色 被引 615 · OpenAlex

本文提出了首个用于睡眠阶段分类的深度学习方法,无需计算频谱图或提取手工特征即可端到端学习,利用了全部多变量多模态 PSG 信号(EEG、EMG、EOG),并能利用每个 30 秒窗口数据的时序上下文。This work introduces here the first deep learning approach for sleep stage classification that learns end-to-end without computing spectrograms or extracting handcrafted features, that exploits all multivariate and multimodal polysomnography (PSG) signals (EEG, EMG, and EOG), and that can exploit the temporal context of each 30-s window of data.

Transformer in Transformer
Transformer in Transformer
arXiv:2103.00112 多模态 方法 OA · 绿色 被引 2262 · S2

本工作指出,这些局部 patch 内部的注意力同样是构建高性能视觉 Transformer 的关键,并探索了一种新架构,即 Transformer iN Transformer (TNT)。It is pointed out that the attention inside these local patches are also essential for building visual transformers with high performance and a new architecture, namely, Transformer iN Transformer (TNT), is explored.

Representation Learning with Contrastive Predictive Coding
基于对比预测编码的表征学习
arXiv:1807.03748 多模态 方法 OA · 绿色 被引 14265 · S2

本文提出了一种通用的无监督学习方法——对比预测编码(Contrastive Predictive Coding),用于从高维数据中提取有用的表征,并在语音、图像、文本和 3D 环境中的强化学习四个不同领域取得了出色的性能。This work proposes a universal unsupervised learning approach to extract useful representations from high-dimensional data, which it calls Contrastive Predictive Coding, and demonstrates that the approach is able to learn useful representations achieving strong performance on four distinct domains: speech, images, text and reinforcement learning in 3D environments.

Diffusion-Convolutional Neural Networks
扩散卷积神经网络
arXiv:1511.02136 多模态 方法 OA · 绿色 被引 1379 · S2

通过引入扩散卷积运算,本文展示了如何从图结构数据中学习基于扩散的表示,并将其作为节点分类的有效基础。Through the introduction of a diffusion-convolution operation, it is shown how diffusion-based representations can be learned from graph-structured data and used as an effective basis for node classification.

Florence: A New Foundation Model for Computer Vision
Florence:面向计算机视觉的新基础模型
arXiv:2111.11432 多模态 方法 OA · 绿色 被引 1152 · S2

本文提出新的计算机视觉基础模型 Florence,通过融入来自 Web 规模图文数据的通用视觉-语言表示,将表征范围从粗粒度(场景)扩展到细粒度、从静态(图像)扩展到动态(视频)、从 RGB 扩展到多种模态(描述、深度等)。This work introduces a new computer vision foundation model, Florence, to expand the representations from coarse (scene) to fine, from static (images) to dynamic (videos), and from RGB to multiple modalities (caption, depth), by incorporating universal visual-language representations from Web-scale image-text data.

Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networks
使用正则化深度 LSTM 网络进行基于骨骼动作识别的共现特征学习
arXiv:1603.07772 多模态 应用落地 OA · 绿色 被引 930 · S2

本文在每个时间步以骨骼作为输入,引入一种新的正则化方案来学习骨骼关节的共现特征,并提出一种同时作用于 LSTM 神经元门、单元和输出响应的新型 dropout 算法。This work takes the skeleton as the input at each time slot and introduces a novel regularization scheme to learn the co-occurrence features of skeleton joints, and proposes a new dropout algorithm which simultaneously operates on the gates, cells, and output responses of the LSTM neurons.

Domain Adaptation for Visual Applications: A Comprehensive Survey
视觉应用中的域适应:综合综述
arXiv:1702.05374 多模态 综述 OA · 绿色 被引 554 · S2

综述领域自适应与迁移学习,重点关注视觉应用及超越图像分类的方法,如目标检测、图像分割、视频分析或视觉属性学习。An overview of domain adaptation and transfer learning with a specific view on visual applications and the methods that go beyond image categorization, such as object detection or image segmentation, video analyses or learning visual attributes are overviewed.

Robust Speech Recognition via Large-Scale Weak Supervision
Robust Speech Recognition via Large-Scale Weak Supervision
arXiv:2212.04356 多模态 方法 OA · 绿色 被引 8184 · S2

当将监督规模扩展到 680,000 小时的多语言、多任务数据时,所得到的模型在标准 benchmark 上泛化良好,在 zero-shot transfer 设置下常可与此前全监督方法的结果相当,且无需任何微调。When scaled to 680,000 hours of multilingual and multitask supervision, the resulting models generalize well to standard benchmarks and are often competitive with prior fully supervised results but in a zero-shot transfer setting without the need for any fine-tuning.