介绍 Gemini for Google (GfG)，一款面向 Google 内部软件工程生态的 Gemini 专用适配版本，涵盖从构建万亿 token 的专有数据集到采用可缓解灾难性遗忘的中段训练策略的完整过程。G Gemini for Google (GfG)}, an adaptation of Gemini specialized for Google's internal software engineering ecosystem, is introduced, from curating a trillion-token proprietary dataset to implementing a mid-training strategy that mitigates catastrophic forgetting.
论文
8 张论文卡片 · 工程化 · 方法
条目 F: Google 企业定制 LLM — 代码转换实战数据
⑥ "How are MLOps Frameworks Used in Open Source Projects"（arXiv:2601.18591）
⑥ "How are MLOps Frameworks Used in Open Source Projects"（arXiv:2601.18591）
对八款主流开源 MLOps 框架的实践使用与功能增强需求进行调查，结果显示 MLOps 框架很少被直接开箱即用，也较少集成进 GitHub Workflows，开发者更多通过其 API 在项目中实现自定义功能。Investigating the practical use and desired feature enhancements of eight popular open-source MLOps frameworks indicates that users mainly ask for enhancements to core features of the frameworks, but also better API exposure and CI/CD integration.
[DataEvolver] Automatic Data Preparation for Large Language Models through Multi-Level Self-Evolving
DataEvolver：基于多层级自演化的 LLM 自动化数据准备
实验表明，DataEvolver 显著提升了数据质量，相比在原始数据上训练，下游 LLM 性能平均提升 10%，凸显了 LLM 与数据迭代协同演化的新机遇。Experiments show that DataEvolver substantially improves data quality and achieves an average 10\% gain in downstream LLM performance compared with training on original data, highlighting new opportunities for the iterative co-evolution of LLMs and data.
4️⃣ arXiv · LoRAFusion（⭐⭐⭐ 值得追踪）
4️⃣ arXiv · LoRAFusion（⭐⭐⭐ 值得追踪）
本文提出 LoRAFusion，一种面向 LLM 的高效 LoRA 微调系统，可消除不必要的内存访问，在不付出重算或同步代价的前提下保持 compute-bound GEMM 的性能，并引入面向多任务微调的自适应批处理算法。LoRAFusion is introduced, an efficient LoRA fine-tuning system for LLMs that eliminates unnecessary memory accesses and preserves the performance of compute-bound GEMMs without incurring the cost of recomputation or synchronization and introduces an adaptive batching algorithm for multi-job fine-tuning.
3️⃣ arXiv · MatryoshkaLoRA（⭐⭐⭐⭐ 值得关注）
3️⃣ arXiv · MatryoshkaLoRA（⭐⭐⭐⭐ 值得关注）
本文提出 MatryoshkaLoRA，一种受 Matryoshka 启发、面向 LoRA 的通用训练框架，通过在已有 LoRA adapter 之间插入一个固定的、经精心设计的对角矩阵来按比例缩放其子秩，从而学习到准确的层次化低秩表示。MatryoshkaLoRA is proposed, a general, Matryoshka-inspired training framework for LoRA that learns accurate hierarchical low-rank representations by inserting a fixed, carefully crafted diagonal matrix between the existing LoRA adapters to scale their sub-ranks accordingly.
3. Kubernetes for GenAI Inference（arXiv:2602.04900v2）
Kubernetes for GenAI Inference（arXiv:2602.04900v2）
这些结果表明，Kueue、DAS 与 GAIE 等互补组件构成了一个高性能的协同平台，证明了 Kubernetes 能够作为承载高要求 GenAI 工作负载的统一底座。These findings illustrate that these complementary components (Kueue, DAS, and GAIE) form a cohesive, high-performance platform, proving Kubernetes' capability to serve as a unified foundation for demanding GenAI workloads.
Multitask Prompted Training Enables Zero-Shot Task Generalization
多任务提示训练实现零样本任务泛化
一个能够将任意自然语言任务轻松映射为人类可读 prompt 形式的系统，并在覆盖多种任务的多任务混合数据上对预训练 encoder-decoder 模型进行微调。A system for easily mapping any natural language tasks into a human-readable prompted form and fine-tune a pretrained encoder-decoder model on this multitask mixture covering a wide variety of tasks.
Improved Training of Wasserstein GANs
Improved Training of Wasserstein GANs
本文提出一种权重裁剪的替代方案：对 critic 相对于其输入的梯度范数施加惩罚。其性能优于标准 WGAN，能以几乎无需调参的方式稳定训练多种 GAN 架构。This work proposes an alternative to clipping weights: penalize the norm of gradient of the critic with respect to its input, which performs better than standard WGAN and enables stable training of a wide variety of GAN architectures with almost no hyperparameter tuning.