结果表明,剪枝效果更取决于剪枝应用的位置,而非具体的评分规则:早期剪枝带来最大的端到端节省,后期剪枝主要用于细化最终的合成上下文。The results show that pruning effectiveness depends more on where pruning is applied than on the specific scoring rule: early pruning yields the largest end-to-end savings, while later pruning mainly refines the final synthesis context.
论文
711 张论文卡片 · 方法
iFAN 提出 Adjusted Probability-Mask Ranking (APMR),将查询竞争与预测的掩码质量对齐,抑制高置信度但不准确的竞争者;同时采用 Cross-Layer Self-Distillation (CLSD) 将更强的中间预测传递至最终层。iFAN introduces Adjusted Probability-Mask Ranking (APMR), which aligns query competition with predicted mask quality and suppresses high-confidence but inaccurate competitors, and employs Cross-Layer Self-Distillation (CLSD) to transfer stronger intermediate predictions to the final layer.
本文引入静止状态 (rest-state) 公式化方法,从单一闭合构型重建关节物体——这是一种固有的不适定设定,几何、语义与运动先验在此弥补运动线索的缺失。This work introduces a rest-state formulation that reconstructs articulated objects from a single closed configuration, an inherently ill-posed setting where geometry, semantics, and motion priors compensate for the absence of motion cues.
本文提出一种自我知识 RAG 框架,引导 LLM 从专利匹配查询中自主提取关键技术实体并构建层次化本体结构,从而实现查询扩展与精确检索。A self-knowledge RAG framework is proposed that guides LLMs to autonomously extract key technical entities and construct hierarchical ontological structures from patent matching queries, thereby enabling query expansion and precise retrieval.
本文提出 InSight-doc,一种智能体视觉感知框架,将视觉分辨率视为一种自适应的推理时资源,从低分辨率起步,选择性地放大高分辨率区域以获取更细粒度的证据,且不依赖任何外部检索器。This work proposes InSight-doc, an agentic visual perception framework that treats visual resolution as an adaptive reasoning-time resource that starts from low resolution and selectively zooms into high-resolution regions for finer evidence, without relying on any external retriever.
本工作提出在海量文本语料上使用新的自监督目标 PEGASUS 对大型 Transformer 编码器-解码器模型进行预训练,并证明其在所有 12 个下游数据集上按 ROUGE 分数衡量均取得 SOTA 性能This work proposes pre-training large Transformer-based encoder-decoder models on massive text corpora with a new self-supervised objective, PEGASUS, and demonstrates it achieves state-of-the-art performance on all 12 downstream datasets measured by ROUGE scores.
尽管方法简单,Decision Transformer 在 Atari、OpenAI Gym 和 Key-to-Door 任务上达到或超过 SOTA 无模型离线 RL 基线的性能Despite its simplicity, Decision Transformer matches or exceeds the performance of state-of-the-art model-free offline RL baselines on Atari, OpenAI Gym, and Key-to-Door tasks.
提出了能够平衡"延迟告警的上下文敏感成本"与"打断成本"的模型与推理流程,并通过对用户活动与通知内容的分析,描述了在不确定性下推理此类成本所面临的挑战Models and inference procedures that balance the context-sensitive costs of deferring alerts with the cost of interruption are presented and the challenge of reasoning about such costs under uncertainty via an analysis of user activity and the content of notifications is described.
自组织临界性作为一种具有内在序参数的唯象框架登场,使开放性主张得以转化为可证伪的猜想。Self-organized criticality enters as a phenomenological framework with an intrinsic order parameter; open claims become falsifiable conjectures.
本文提出连续性核 (Continuity Kernel, CK),一种激活契约,将提交前候选评估与原子状态激活解耦,将连续性定义为已接受分支头的连续且经过授权的谱系。The Continuity Kernel (CK), an activation contract that decouples off-commit candidate evaluation from atomic state activation from atomic state activation is presented, defining continuity as an unbroken, authorized lineage of accepted branch heads.
本文提出一种框架,用于从系统描述自动构建 DML 模型并将其表示为知识图谱 (KG-DML),以 RAG 和 LLM 作为使能工具。This study presents a framework for automated construction of DML models from system descriptions and their representation as Knowledge Graphs (KG-DML), using Retrieval-Augmented Generation and Large Language Models as enabling tools.
结果表明,长周期软件开发可以围绕持久化项目而非持久化智能体来组织;本文提出 EvoX Genesis,使软件项目保持持久,同时允许局部智能体保持有限生命周期。Results show that long-horizon software development can be organized around a persistent project rather than a persistent agent, and EvoX Genesis is introduced, which instead makes the software project persistent while allowing local agents to remain finite-lived.
本文表明,利用在大规模数据上预训练的 HPE 模型先验知识作为骨干网络可在该任务上取得高性能,并验证了手部可见性检测器在通过 2D 关键点多视角三角化进行 3D 手部姿态标注的下游任务中的有效性。It is shown that leveraging the prior knowledge of HPE models pretrained on large-scale data as a backbone yields high performance in this task, and the utility of Hand Visibility Detector on a downstream task of 3D hand pose annotation via multi-view triangulation of 2D keypoints.
AtlasVLA 是一种新框架,通过持久化的世界-自我状态从直接反应式操作转向主动推理,显著优于多视角基线,在 LIBERO-Long 上取得 9.4% 的绝对成功率提升,在真实世界长周期任务中取得 17.5% 的提升。AtlasVLA is a novel framework that transitions from direct reactive manipulation to proactive reasoning through a persistent world-ego state and decisively outperforms multi-view baselines, yielding absolute success rate improvements of 9.4% on LIBERO-Long and 17.5% in real-world long-horizon tasks.
尽管聚合准确率有所提升,但视觉工具使用在广泛的 rollout 中并不具备因果有效性。Despite aggregate accuracy gains, visual tool-use is not causally effective across a broad range of rollouts: despite aggregate accuracy gains, visual tool-use is not causally effective across a broad range of rollouts.
Mechanist 是一个 Agentic 系统,将 AI 作为科学仪器用于自主发现 AI 智能背后的机制,并发展了关于 belief 的机制理论,揭示了模型如何表征世界知识、形成 belief、推断他人 belief,以及这些机制如何在预训练过程中涌现。Mechanist is an agentic system that uses AI as a scientific instrument for the autonomous discovery of mechanisms underlying AI intelligence, and develops a mechanism theory of belief, revealing how models represent world knowledge, form beliefs, infer the beliefs of others, and how these mechanisms emerge during pretraining.
两项研究为 GPU Agent 控制建立了两个可度量的门槛:deadline 可达的 cohort 供给与观测放置,并使用固定分区份额 F、精确离线份额 P*、局部上界 U 和在线达成份额 A 对 ready-cohort 边界进行了形式化。Two studies establish two measurable gates for GPU agent control: deadline-feasible cohort supply and observation placement and formalize the ready-cohort boundary using fixed-partition share F, exact offline share P*, local upper bound U, and online achieved share A.
本文提出 SHAPER,一种免训练具身自适应的自演化框架,保持模型参数冻结,通过目标环境 rollout 演化可复用的 skills 和 context-code harness 来改进非参数化 Agent 系统。This work proposes SHAPER, a self-evolving framework for train-free embodied adaptation that keeps model parameters frozen and improves the non-parametric agent system by evolving reusable skills and a context-code harness through target-environment rollouts.
本文提出 Simplax,一种精确的 Dirichlet-类别增强方法,将每个被破坏的类别状态与一个辅助的 simplex 值变量耦合,同时保留原始均匀扩散过程作为其类别边缘分布。Simplax is introduced, an exact Dirichlet--categorical augmentation that couples each corrupted categorical state with an auxiliary simplex-valued variable while preserving the original uniform diffusion process as its categorical marginal.
本文提供了参数空间探索能够改进 LLM 强化学习的证据,并提出称为扰动参数策略优化(3PO)的方法族,使用不同的采样策略和不同的 rollout 分组进行 reward 估计。Evidence that parameter-space exploration can improve reinforcement learning for LLMs is presented, and a family of methods called Perturbed Parameter Policy Optimization (3PO) is introduced which use different sampling strategies and different rollout grouping for reward estimation.
本文提出 SkillZip,一种执行感知的程序化抽象框架,对 section 级图执行保持契约的压缩,加载一个紧凑、依赖闭合的 context,并仅在需要时展开宏。SkillZip is proposed, an execution-aware procedural abstraction framework that performs contract-preserving compression over section-level graphs that hydrates a compact, dependency-closed context and expands macros only when required.
一个仅依赖字符级输入的简单神经语言模型,仅从字符即可编码语义和正字法信息,表明在许多语言中,字符输入足以完成语言建模。A simple neural language model that relies only on character-level inputs that is able to encode, from characters only, both semantic and orthographic information and suggests that on many languages, character inputs are sufficient for language modeling.
描述了一种无监督学习通用分布式句子编码器的方法,利用书籍文本的连续性,训练编码器-解码器模型以重建编码段落的周围句子。The approach for unsupervised learning of a generic, distributed sentence encoder is described, using the continuity of text from books to train an encoder-decoder model that tries to reconstruct the surrounding sentences of an encoded passage.
本工作训练了一个预测的计算最优模型 Chinchilla,使用与 Gopher 相同的计算预算,但参数量为 70B、数据量为 4 倍,达到 SOTA 平均准确率,比 Gopher 提升超过 7%。This work trains a predicted compute-optimal model, Chinchilla, that uses the same compute budget as Gopher but with 70B parameters and 4$\times$ more more data, and reaches a state-of-the-art average accuracy, greater than a 7% improvement over Gopher.
本工作综述了可用于推断用户需求的贝叶斯用户模型研究,这些模型综合考虑用户的背景、操作和查询,并提出了一种智能用户界面的整体架构。This work reviews work on Bayesian user models that can be employed to infer a user's needs by considering a users' background, actions, and queries and proposes an overall architecture for an intelligent user interface.
探索以交错方式使用 LLM 同时生成推理轨迹和任务特定动作,使两者产生更大协同:推理轨迹帮助模型归纳、跟踪和更新动作计划以及处理异常,而动作使其与外部源交互以获取额外信息。The use of LLMs are explored to generate both reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two: reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources to gather additional information.
本文提出提示式注视目标估计(PGE)任务,一种用于注视分析的端到端、概念驱动新范式,并推出首个为 PGE 设计的模型 GazeAnywhere,它使用基于 Transformer 的检测器融合冻结编码器特征,同时解决主体定位、画内/画外存在性以及注视目标热图估计。The Promptable Gaze Target Estimation (PGE) task is introduced, a new end-to-end, concept-driven paradigm for gaze analysis and GazeAnywhere, the first model designed for PGE, uses a transformer-based detector to fuse features from frozen encoders and simultaneously solves subject localization, in/out-of-frame presence, and gaze target heatmap estimation.
ParliamentRAG 是一种主题相关的权威模型,根据当前 query 估计每位发言者的权威性,结合职业、教育和此前发言等可解释组件,以应对政治敏感文本中最高频发言者主导、无法按主题专长加权发言者以及引用归属错误的风险。ParliamentRAG is a topic-dependent authority model that estimates each speaker's authority as a function of the current query, combining interpretable components such as profession, education, and previous interventions that addresses risks of dominance of the most frequent speakers, inability to weight speakers according to topical expertise, and citation misattribution in politically sensitive text.
本文提出一种用于机器人操作的动作条件视频世界模型,给定观测帧、语言指令以及由末端执行器位姿和夹爪状态组成的预定动作序列,预测对应的未来观测结果。An action-conditioned video world model for robotic manipulation that, given an observed frame, a language instruction, and a prescribed action sequence comprising end-effector poses and gripper states, predicts the resulting future observations is presented.
本文提出 UniSwap,这是首个用于说话视频中流式联合音视频身份替换的框架,并引入 swap-and-reconstruct 流程,从真实片段中移除视觉和声音身份,同时使用原始片段作为重建目标。This work presents UniSwap, the first framework for streaming joint audio-visual identity replacement in talking videos, and introduces a swap-and-reconstruct pipeline that removes visual and vocal identity from real clips and uses the original clips as reconstruction targets.
本文认为关键在于设计合适的 search-state 表示,将 LLM 的内部知识与 visual-token 剪枝的结构要求和约束相连接,并提出 AutoPrune,一种用于 LLM 驱动的 visual-token 剪枝策略设计的免训练框架。This paper argues that the key lies in designing an appropriate search-state representation that connects the internal knowledge of LLMs with the structural requirements and constraints of visual-token pruning, and proposes AutoPrune, a training-free framework for LLM-driven visual-token pruning policy design.
联合部署可将攻击成功率从 67.4% 降至 14.0%,同时在未投毒检索上保留 41.3% 的 F1,证明无需投毒标签或可信语料库即可在语料入库和查询时提供实用的双重保护。Joint deployment reduces attack success from 67.4% to 14.0% while retaining 41.3% F1 on unpoisoned retrieval, demonstrating practical protection at both corpus ingestion and query time without poison labels or trusted corpora.
本文提出 Reduced Matrix Multiplication,一种免训练的输入自适应推理方法,通过沿收缩维度选取信息性切片来减少 Transformer 矩阵乘积,且不修改模型权重,并表明同一原理可扩展到多模态视觉-语言推理。Reduced Matrix Multiplication is proposed, a training-free, input-adaptive inference method that reduces Transformer matrix products by selecting informative slices along their contraction dimensions, without modifying model weights, and it is shown that the same principle extends to multimodal vision-language inference.
提出 Agentic Video Auto-Encoder(AVA-Encoder),一种由 agentic 自我进化驱动的新型自编码框架,用于学习 agent-native 视频表示,在 shot-level 和 keyframe-level system-prompt token 使用量减少 74.3% 的同时,性能优于精心人工调优的策略。The Agentic Video Auto-Encoder (AVA-Encoder), a novel auto-encoding framework driven by agentic self-evolution to learn agent-native video representations that outperforms a carefully human-tuned policy while using 74.3% fewer shot-level and keyframe-level system-prompt tokens.
论文表明,使用 Stanford Natural Language Inference 数据集有监督训练的通用句子表示,在广泛的迁移任务上能持续优于 SkipThought vectors 等无监督方法。It is shown how universal sentence representations trained using the supervised data of the Stanford Natural Language Inference datasets can consistently outperform unsupervised methods like SkipThought vectors on a wide range of transfer tasks.
提出 Context-Matched Distillation(CMD),一种因果 DMD 框架,将 teacher 监督信号与每个 target 生成时可用信息对齐,并可自然扩展到逐帧和逐 chunk 生成、长视频蒸馏以及相机条件蒸馏。Context-Matched Distillation (CMD) is introduced, a causal DMD framework that aligns teacher supervision with the information available when each target is generated, and naturally extends to frame-wise and chunk-wise generation, long video distillation, and camera-conditioned distillation.