论文

一个仅依赖字符级输入的简单神经语言模型，仅从字符即可编码语义和正字法信息，表明在许多语言中，字符输入足以完成语言建模。A simple neural language model that relies only on character-level inputs that is able to encode, from characters only, both semantic and orthographic information and suggests that on many languages, character inputs are sufficient for language modeling.

描述了一种无监督学习通用分布式句子编码器的方法，利用书籍文本的连续性，训练编码器-解码器模型以重建编码段落的周围句子。The approach for unsupervised learning of a generic, distributed sentence encoder is described, using the continuity of text from books to train an encoder-decoder model that tries to reconstruct the surrounding sentences of an encoded passage.

本文提出 Reduced Matrix Multiplication，一种免训练的输入自适应推理方法，通过沿收缩维度选取信息性切片来减少 Transformer 矩阵乘积，且不修改模型权重，并表明同一原理可扩展到多模态视觉-语言推理。Reduced Matrix Multiplication is proposed, a training-free, input-adaptive inference method that reduces Transformer matrix products by selecting informative slices along their contraction dimensions, without modifying model weights, and it is shown that the same principle extends to multimodal vision-language inference.

论文表明，使用 Stanford Natural Language Inference 数据集有监督训练的通用句子表示，在广泛的迁移任务上能持续优于 SkipThought vectors 等无监督方法。It is shown how universal sentence representations trained using the supervised data of the Stanford Natural Language Inference datasets can consistently outperform unsupervised methods like SkipThought vectors on a wide range of transfer tasks.

提出 HPSE，构建一种混合 rollout，能够在 student 自身轨迹中 coverage 缺失的精确位置补齐缺失事实，同时在其他位置保持 on-policy。HPSE is proposed, which builds a hybrid rollout that steps in to place missing facts onto the student's own trajectory precisely where its coverage fails, while staying on-policy elsewhere.

Gambit 通过周期性剪除低潜力轨迹并立即从高质量前缀分支，利用轻量 scorer 探测 hidden state，将算力动态集中到最有潜力的推理路径上，同时保持持续的高硬件利用率。By periodically pruning unpromising trajectories and immediately branching from high-quality prefixes, Gambit dynamically concentrates compute onto the most promising reasoning traces via a light-weight scorer probing hidden states while maintaining continuous high hardware utilization.

通过跨 46 个任务（涵盖四个认知领域）的电路分析，发现 LLM 发展出与人类大脑相似的模块化架构：在人类大脑中依赖同一网络的任务会在 LLM 中招募重叠的神经元，而依赖不同网络的任务则招募不同的神经元。Using circuit analyses across N=46 tasks spanning four cognitive domains, it is found that Large Language Models develop a modular architecture that mirrors the human brain: tasks drawing on the same network in humans recruit overlapping neurons in LLMs, whereas tasks drawing on different networks recruit distinct neurons.

FreeToken 是一个 edge-native 的 MoE serving 系统，将个人机器视为一个统一、弹性的推理平台而非小型 GPU，把开放权重转化为可部署的本地软件，使用户已有的机器成为运行前沿规模智能的实用平台。FreeToken is an edge-native MoE serving system that treats a personal machine not as a small GPU, but as a unified, elastic inference platform, turning open weights into deployable local software, making the machines users already own a practical platform for frontier-scale intelligence.