论文

本文提出了一种双时间尺度更新规则（TTUR），用于在任意 GAN 损失函数下使用 SGD 训练 GAN，并引入了 Frechet Inception Distance（FID），相比 Inception Score 能更好地捕捉生成图像与真实图像之间的相似性。This work proposes a two time-scale update rule (TTUR) for training GANs with stochastic gradient descent on arbitrary GAN loss functions and introduces the "Frechet Inception Distance" (FID) which captures the similarity of generated images to real ones better than the Inception Score.

名为 PaLM 的 540 亿参数、密集激活的 Transformer 语言模型取得了突破性性能，在一系列多步推理任务上超越了微调后的 SOTA，并在最近发布的 BIG-bench 基准上超越了人类平均水平。A 540-billion parameter, densely activated, Transformer language model, which is called PaLM achieves breakthrough performance, outperforming the finetuned state-of-the-art on a suite of multi-step reasoning tasks, and outperforming average human performance on the recently released BIG-bench benchmark.

研究表明，APE 生成的提示词既可引导模型趋向真实性和/或信息量，也可通过将其前置拼接到标准上下文学习提示词之前来提升少样本学习性能。It is shown that APE-engineered prompts can be applied to steer models toward truthfulness and/or informativeness, as well as to improve few-shot learning performance by simply prepending them to standard in-context learning prompts.

论文证明，使用标注数据进行微调，并允许模型查询外部知识源，能够在安全性和事实性这两个关键挑战上带来显著提升。It is demonstrated that fine-tuning with annotated data and enabling the model to consult external knowledge sources can lead to significant improvements towards the two key challenges of safety and factual grounding.

研究表明 BigBird 是序列函数的通用逼近器，且具备图灵完备性，从而保留了二次全注意力模型的这些性质。It is shown that BigBird is a universal approximator of sequence functions and is Turing complete, thereby preserving these properties of the quadratic, full attention model.

本文提出 OasisKV，一种以显存为中心的 LLM 推理系统设计，通过在 LLM 解码期间将完整 KV-cache 存储与 HBM 解耦来缓解 HBM 容量压力，并观察到未来重要 token 可借助推测解码（SD）所起草的前瞻 token 被提前准确预测。OasisKV is presented, a memory-centric LLM inference system design that alleviates HBM capacity pressure by decoupling full KV-cache storage from HBM during LLM decoding and observes that future important tokens can be predicted accurately in advance using lookahead tokens drafted by speculative decoding (SD).

一个结构定理将无记忆等变规则刻画为恰好由 Gram 矩阵决定的左预处理子；一个迁移定理将梯度流的路径性质推广到 common-scalar 流。A structure theorem characterizes the memoryless equivariant rules as exactly the Gram-determined left preconditioners, and a transfer theorem carries gradient flow's pathwise properties to common-scalar flows.

本工作研究图结构输入的特征学习技术，并在程序验证任务上取得 SOTA 性能，该任务需将子图与抽象数据结构进行匹配。This work studies feature learning techniques for graph-structured inputs and achieves state-of-the-art performance on a problem from program verification, in which subgraphs need to be matched to abstract data structures.

iFAN 提出 Adjusted Probability-Mask Ranking (APMR)，将查询竞争与预测的掩码质量对齐，抑制高置信度但不准确的竞争者；同时采用 Cross-Layer Self-Distillation (CLSD) 将更强的中间预测传递至最终层。iFAN introduces Adjusted Probability-Mask Ranking (APMR), which aligns query competition with predicted mask quality and suppresses high-confidence but inaccurate competitors, and employs Cross-Layer Self-Distillation (CLSD) to transfer stronger intermediate predictions to the final layer.

自组织临界性作为一种具有内在序参数的唯象框架登场，使开放性主张得以转化为可证伪的猜想。Self-organized criticality enters as a phenomenological framework with an intrinsic order parameter; open claims become falsifiable conjectures.

一个仅依赖字符级输入的简单神经语言模型，仅从字符即可编码语义和正字法信息，表明在许多语言中，字符输入足以完成语言建模。A simple neural language model that relies only on character-level inputs that is able to encode, from characters only, both semantic and orthographic information and suggests that on many languages, character inputs are sufficient for language modeling.

描述了一种无监督学习通用分布式句子编码器的方法，利用书籍文本的连续性，训练编码器-解码器模型以重建编码段落的周围句子。The approach for unsupervised learning of a generic, distributed sentence encoder is described, using the continuity of text from books to train an encoder-decoder model that tries to reconstruct the surrounding sentences of an encoded passage.

本文提出 Reduced Matrix Multiplication，一种免训练的输入自适应推理方法，通过沿收缩维度选取信息性切片来减少 Transformer 矩阵乘积，且不修改模型权重，并表明同一原理可扩展到多模态视觉-语言推理。Reduced Matrix Multiplication is proposed, a training-free, input-adaptive inference method that reduces Transformer matrix products by selecting informative slices along their contraction dimensions, without modifying model weights, and it is shown that the same principle extends to multimodal vision-language inference.

论文表明，使用 Stanford Natural Language Inference 数据集有监督训练的通用句子表示，在广泛的迁移任务上能持续优于 SkipThought vectors 等无监督方法。It is shown how universal sentence representations trained using the supervised data of the Stanford Natural Language Inference datasets can consistently outperform unsupervised methods like SkipThought vectors on a wide range of transfer tasks.

提出 HPSE，构建一种混合 rollout，能够在 student 自身轨迹中 coverage 缺失的精确位置补齐缺失事实，同时在其他位置保持 on-policy。HPSE is proposed, which builds a hybrid rollout that steps in to place missing facts onto the student's own trajectory precisely where its coverage fails, while staying on-policy elsewhere.

Gambit 通过周期性剪除低潜力轨迹并立即从高质量前缀分支，利用轻量 scorer 探测 hidden state，将算力动态集中到最有潜力的推理路径上，同时保持持续的高硬件利用率。By periodically pruning unpromising trajectories and immediately branching from high-quality prefixes, Gambit dynamically concentrates compute onto the most promising reasoning traces via a light-weight scorer probing hidden states while maintaining continuous high hardware utilization.

通过跨 46 个任务（涵盖四个认知领域）的电路分析，发现 LLM 发展出与人类大脑相似的模块化架构：在人类大脑中依赖同一网络的任务会在 LLM 中招募重叠的神经元，而依赖不同网络的任务则招募不同的神经元。Using circuit analyses across N=46 tasks spanning four cognitive domains, it is found that Large Language Models develop a modular architecture that mirrors the human brain: tasks drawing on the same network in humans recruit overlapping neurons in LLMs, whereas tasks drawing on different networks recruit distinct neurons.

FreeToken 是一个 edge-native 的 MoE serving 系统，将个人机器视为一个统一、弹性的推理平台而非小型 GPU，把开放权重转化为可部署的本地软件，使用户已有的机器成为运行前沿规模智能的实用平台。FreeToken is an edge-native MoE serving system that treats a personal machine not as a small GPU, but as a unified, elastic inference platform, turning open weights into deployable local software, making the machines users already own a practical platform for frontier-scale intelligence.