活文档 · 论文卡 · 笔记 · 仓库 · 攻略
这项系统性研究在数十项语言理解任务上比较了预训练目标、架构、无标注数据集、迁移方法及其他因素，并在涵盖摘要、问答、文本分类等的许多基准上取得了 SOTA 结果。This systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks and achieves state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more.
提出面向检索增强生成（RAG）的通用微调方案——RAG 模型融合预训练参数化记忆与非参数化记忆进行语言生成；研究发现，相较 SOTA 的纯参数化 seq2seq 基线，RAG 模型生成的文本更具针对性、更多样且更符合事实。A general-purpose fine-tuning recipe for retrieval-augmented generation (RAG) -- models which combine pre-trained parametric and non-parametric memory for language generation, and finds that RAG models generate more specific, diverse and factual language than a state-of-the-art parametric-only seq2seq baseline.
本文开发并发布了 Llama 2，这是一系列参数规模从 70 亿到 700 亿不等的预训练与微调大语言模型（LLMs），有望成为闭源模型的合适替代品。This work develops and releases Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters, which may be a suitable substitute for closed-source models.
探索以交错方式使用 LLM 同时生成推理轨迹和任务特定动作，使两者产生更大协同：推理轨迹帮助模型归纳、跟踪和更新动作计划以及处理异常，而动作使其与外部源交互以获取额外信息。The use of LLMs are explored to generate both reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two: reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources to gather additional information.
更大的模型显著更具样本效率，因此最优的算力高效训练方式是：在相对适中的数据量上训练非常大的模型，并在远未收敛时显著提前停止训练。Larger models are significantly more sample-efficient, such that optimally compute-efficient training involves training very large models on a relatively modest amount of data and stopping significantly before convergence.
名为 PaLM 的 540 亿参数、密集激活的 Transformer 语言模型取得了突破性性能，在一系列多步推理任务上超越了微调后的 SOTA，并在最近发布的 BIG-bench 基准上超越了人类平均水平。A 540-billion parameter, densely activated, Transformer language model, which is called PaLM achieves breakthrough performance, outperforming the finetuned state-of-the-art on a suite of multi-step reasoning tasks, and outperforming average human performance on the recently released BIG-bench benchmark.
对现有图神经网络模型进行了详细综述，系统性地归纳了其应用，并提出了四个有待解决的未来研究方向A detailed review over existing graph neural network models is provided, systematically categorize the applications, and four open problems for future research are proposed.
本文详细介绍了 CNN 在多个方面的改进，包括层设计、激活函数、损失函数、正则化、优化与快速计算，并阐述了卷积神经网络在计算机视觉、语音与自然语言处理中的多种应用。This paper details the improvements of CNN on different aspects, including layer design, activation function, loss function, regularization, optimization and fast computation, and introduces various applications of convolutional neural networks in computer vision, speech and natural language processing.
本文提出 Toolformer，训练其决定调用哪些 API、何时调用、传入什么参数，以及如何将结果最佳地融入后续 token 预测，在多种下游任务上显著提升零样本性能。This paper introduces Toolformer, a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction, which achieves substantially improved zero-shot performance across a variety of downstream tasks.
本文推出 ScanNet，一个 RGB-D 视频数据集，包含 1513 个场景中的 250 万视角，标注有三维相机位姿、表面重建与语义分割，并表明使用该数据可在多项三维场景理解任务上取得 SOTA 性能。This work introduces ScanNet, an RGB-D video dataset containing 2.5M views in 1513 scenes annotated with 3D camera poses, surface reconstructions, and semantic segmentations, and shows that using this data helps achieve state-of-the-art performance on several 3D scene understanding tasks.
一种面向语言模型推理的新框架 Tree of Thoughts (ToT)，推广了流行的 Chain of Thought 提示方法，允许在作为问题求解中间步骤的连贯文本单元（thoughts）上进行探索。A new framework for language model inference, Tree of Thoughts (ToT), which generalizes over the popular Chain of Thought approach to prompting language models, and enables exploration over coherent units of text (thoughts) that serve as intermediate steps toward problem solving.
简化了 MoE 路由算法，设计出通信与计算成本更低的直观改进模型，并首次证明大型稀疏模型可以使用更低精度格式进行训练This work simplifies the MoE routing algorithm and design intuitive improved models with reduced communication and computational costs and shows large sparse models may be trained, for the first time, with lower precision formats.
AI 编程助手 Skill（兼容 Claude Code、Codex、OpenCode、Cursor、Gemini CLI 等），可将任意代码、SQL schema、R 脚本、shell 脚本、文档、论文、图片或视频文件夹转换为可查询的知识图谱，应用代码、数据库 schema 与基础设施统一于一张图谱中。AI coding assistant skill (Claude Code, Codex, OpenCode, Cursor, Gemini CLI, and more). Turn any folder of code, SQL schemas, R scripts, shell scripts, docs, papers, images, or videos into a queryable knowledge graph. App code + database schema + infrastructure in one graph.
为每个 Agent 提供跨会话持久上下文——捕获会话中 Agent 的所有行为，经 AI 压缩后注入到未来会话中。支持 Claude Code、OpenClaw、Codex、Gemini、Hermes、Copilot、OpenCode 等。Persistent Context Across Sessions for Every Agent – Captures everything your agent does during sessions, compresses it with AI, and injects relevant context back into future sessions. Works with Claude Code, OpenClaw, Codex, Gemini, Hermes, Copilot, OpenCode + More
开源 LLM 知识平台：将原始文档转化为可查询的 RAG、自主推理 agent 和可自维护的 Wiki。Open-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki.
《深入理解 AI Agent：设计原理与工程实践》（李博杰 著）开源主仓库：全书正文、编译版 PDF 与按章配套代码
将任何代码库及其文档、SQL schema、配置文件和 PDF 转化为可查询的知识图谱。适用于 Claude Code、Cursor、Codex 和 Gemini CLI 的 /graphify skill：本地确定性 AST 解析，每条边都有解释，无需向量存储。Turn any codebase, with its docs, SQL schemas, configs, and PDFs, into a queryable knowledge graph. A /graphify skill for Claude Code, Cursor, Codex, and Gemini CLI: local deterministic AST parsing, every edge explained, no vector store.
DeepTutor:Agent 原生的个性化辅导系统。https://deeptutor.info/DeepTutor: Lifelong Personalized Tutoring. https://deeptutor.info/.