介绍 Gemini for Google (GfG)，一款面向 Google 内部软件工程生态的 Gemini 专用适配版本，涵盖从构建万亿 token 的专有数据集到采用可缓解灾难性遗忘的中段训练策略的完整过程。G Gemini for Google (GfG)}, an adaptation of Gemini specialized for Google's internal software engineering ecosystem, is introduced, from curating a trillion-token proprietary dataset to implementing a mid-training strategy that mitigates catastrophic forgetting.
论文
10 张论文卡片 · 工程化 · 方法
条目 F: Google 企业定制 LLM — 代码转换实战数据
⑥ "How are MLOps Frameworks Used in Open Source Projects"（arXiv:2601.18591）
⑥ "How are MLOps Frameworks Used in Open Source Projects"（arXiv:2601.18591）
对八款主流开源 MLOps 框架的实践使用与功能增强需求进行调查，结果显示 MLOps 框架很少被直接开箱即用，也较少集成进 GitHub Workflows，开发者更多通过其 API 在项目中实现自定义功能。Investigating the practical use and desired feature enhancements of eight popular open-source MLOps frameworks indicates that users mainly ask for enhancements to core features of the frameworks, but also better API exposure and CI/CD integration.
[DataEvolver] Automatic Data Preparation for Large Language Models through Multi-Level Self-Evolving
DataEvolver：基于多层级自演化的 LLM 自动化数据准备
实验表明，DataEvolver 显著提升了数据质量，相比在原始数据上训练，下游 LLM 性能平均提升 10%，凸显了 LLM 与数据迭代协同演化的新机遇。Experiments show that DataEvolver substantially improves data quality and achieves an average 10\% gain in downstream LLM performance compared with training on original data, highlighting new opportunities for the iterative co-evolution of LLMs and data.
4️⃣ arXiv · LoRAFusion（⭐⭐⭐ 值得追踪）
4️⃣ arXiv · LoRAFusion（⭐⭐⭐ 值得追踪）
本文提出 LoRAFusion，一种面向 LLM 的高效 LoRA 微调系统，可消除不必要的内存访问，在不付出重算或同步代价的前提下保持 compute-bound GEMM 的性能，并引入面向多任务微调的自适应批处理算法。LoRAFusion is introduced, an efficient LoRA fine-tuning system for LLMs that eliminates unnecessary memory accesses and preserves the performance of compute-bound GEMMs without incurring the cost of recomputation or synchronization and introduces an adaptive batching algorithm for multi-job fine-tuning.
3️⃣ arXiv · MatryoshkaLoRA（⭐⭐⭐⭐ 值得关注）
3️⃣ arXiv · MatryoshkaLoRA（⭐⭐⭐⭐ 值得关注）
本文提出 MatryoshkaLoRA，一种受 Matryoshka 启发、面向 LoRA 的通用训练框架，通过在已有 LoRA adapter 之间插入一个固定的、经精心设计的对角矩阵来按比例缩放其子秩，从而学习到准确的层次化低秩表示。MatryoshkaLoRA is proposed, a general, Matryoshka-inspired training framework for LoRA that learns accurate hierarchical low-rank representations by inserting a fixed, carefully crafted diagonal matrix between the existing LoRA adapters to scale their sub-ranks accordingly.
3. Kubernetes for GenAI Inference（arXiv:2602.04900v2）
Kubernetes for GenAI Inference（arXiv:2602.04900v2）
这些结果表明，Kueue、DAS 与 GAIE 等互补组件构成了一个高性能的协同平台，证明了 Kubernetes 能够作为承载高要求 GenAI 工作负载的统一底座。These findings illustrate that these complementary components (Kueue, DAS, and GAIE) form a cohesive, high-performance platform, proving Kubernetes' capability to serve as a unified foundation for demanding GenAI workloads.
BERTopic: Neural topic modeling with a class-based TF-IDF procedure
BERTopic：基于类内 TF-IDF 流程的神经主题建模
提出 BERTopic，一种通过开发类内 TF-IDF 变体来提取一致性主题表示，从而扩展主题建模流程的主题模型BERTopic is presented, a topic model that extends the process of topic modeling by extracting coherent topic representation through the development of a class-based variation of TF-IDF.
Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches
通过训练卷积神经网络比较图像块进行立体匹配
提出一种从校正后的图像对中提取深度信息的方法，使用卷积神经网络在小图像块上学习相似性度量，并针对该任务考察了两种网络架构：一种面向速度优化，另一种面向精度优化This work presents a method for extracting depth information from a rectified image pair by learning a similarity measure on small image patches using a convolutional neural network and examines two network architectures for this task: one tuned for speed, the other for accuracy.
Random Erasing Data Augmentation
Random Erasing 数据增强
在训练过程中，Random Erasing 在图像中随机选择一个矩形区域并以随机值擦除其像素，在图像分类、目标检测与行人重识别任务中相较于强基线均带来稳定提升In training, Random Erasing randomly selects a rectangle region in an image and erases its pixels with random values and yields consistent improvement over strong baselines in image classification, object detection and person re-identification.
Training Compute-Optimal Large Language Models
Training Compute-Optimal Large Language Models
本工作训练了一个预测的计算最优模型 Chinchilla，使用与 Gopher 相同的计算预算，但参数量为 70B、数据量为 4 倍，达到 SOTA 平均准确率，比 Gopher 提升超过 7%。This work trains a predicted compute-optimal model, Chinchilla, that uses the same compute budget as Gopher but with 70B parameters and 4$\times$ more more data, and reaches a state-of-the-art average accuracy, greater than a 7% improvement over Gopher.