发现对 GPT 语言模型进行重复采样,是为困难 prompt 生成可用解的一种出乎意料的有效策略,并讨论了部署强大代码生成技术在安全性、安全保障与经济等方面的潜在更广泛影响。It is found that repeated sampling from the GPT language model is a surprisingly effective strategy for producing working solutions to difficult prompts, and the potential broader impacts of deploying powerful code generation technologies, covering safety, security, and economics are discussed.
工程化主题中枢
活文档 · 论文卡 · 笔记 · 仓库 · 攻略
主题活文档 Live Doc
全部论文卡 Papers
全部研究发现,在上述多个维度上进行的指令微调可显著提升多种模型类别(PaLM、T5、U-PaLM)、多种提示设定以及多种评测基准(MMLU、BBH、TyDiQA、MGSM、开放式生成)上的表现。It is found that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups, and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation).
本文对 LLM 的评估方法进行了全面综述,围绕三个关键维度展开:评估什么、在何处评估、如何评估,并为 LLM 评估领域的研究者提供了宝贵洞见。This paper presents a comprehensive review of these evaluation methods for LLMs, focusing on three key dimensions: what to evaluate, where to evaluate, and how to evaluate, and offers invaluable insights to researchers in the realm of LLMs evaluation.
本文显著推进了针对已对齐语言模型的对抗攻击 SOTA,并提出了关于如何防止此类系统生成不良信息的重要问题。This work significantly advances the state-of-the-art in adversarial attacks against aligned language models, raising important questions about how such systems can be prevented from producing objectionable information.
实证研究表明 AutoGen 框架在多个示例应用中有效,应用领域涵盖数学、编码、问答、运筹学、在线决策、娱乐等。Empirical studies demonstrate the effectiveness of the AutoGen framework in many example applications, with domains ranging from mathematics, coding, question answering, operations research, online decision-making, entertainment, etc.
EvalPlus——一个用于严格基准测试 LLM 生成代码功能正确性的代码合成评估框架,通过 LLM 与基于 mutation 的策略驱动的自动测试输入生成器,为给定评估数据集补充大量新生成的测试用例。EvalPlus -- a code synthesis evaluation framework to rigorously benchmark the functional correctness of LLM-synthesized code and augments a given evaluation dataset with large amounts of test-cases newly produced by an automatic test input generator, powered by both LLM and mutation-based strategies.
一篇关于基于 LLM 的 Agent 的全面综述,追溯了 Agent 概念从其哲学起源到在 AI 中的发展历程,解释了为何 LLM 适合作为 Agent 的基础,并提出一个包含三个核心组件的通用框架:大脑、感知与行动。A comprehensive survey on LLM-based agents, tracing the concept of agents from its philosophical origins to its development in AI, and explaining why LLMs are suitable foundations for agents, and presenting a general framework, comprising three main components: brain, perception, and action.
本文描述了一份以模式形式呈现的 prompt 工程技巧目录,这些技巧已被用于解决与 LLMs 对话时的常见问题,以改进 LLM 对话的输出。A catalog of prompt engineering techniques presented in pattern form that have been applied to solve common problems when conversing with LLMs to improve the outputs of LLM conversations is described.
研究表明,APE 生成的提示词既可引导模型趋向真实性和/或信息量,也可通过将其前置拼接到标准上下文学习提示词之前来提升少样本学习性能。It is shown that APE-engineered prompts can be applied to steer models toward truthfulness and/or informativeness, as well as to improve few-shot learning performance by simply prepending them to standard in-context learning prompts.
PaLM 2 是一个新的 SOTA 语言模型,相比其前身 PaLM 具有更强的多语言和推理能力,并具备更高的计算效率,能够在不增加额外开销或影响其他能力的前提下在推理时控制输出毒性。PaLM 2 is a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM and enables inference-time control over toxicity without additional overhead or impact on other capabilities.
提出 BloombergGPT,一个 500 亿参数的语言模型,在广泛的金融数据上训练而成,并基于 Bloomberg 丰富的数据源构建了包含 3630 亿 token 的数据集,可能是迄今最大的领域专用数据集。This work presents BloombergGPT, a 50 billion parameter language model that is trained on a wide range of financial data, and constructs a 363 billion token dataset based on Bloomberg's extensive data sources, perhaps the largest domain-specific dataset yet.
笔记 Notes
全部仓库 Repos
全部面向 AI 编码 Agent 的生产级工程 skillsProduction-grade engineering skills for AI coding agents.
全球首个开源、Agentic 视频制作系统,提供 12 条流水线、52 个工具、500+ agent skills,将 AI 编程助手升级为完整视频制作工作室。World's first open-source, agentic video production system. 12 production pipelines, 100+ tools, 700+ agent skill and production-knowledge files. Turn your AI coding assistant into a full video production studio.
可直接用于生产环境的 Agent 工作流开发平台。Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.
在你机器上运行的求职工具。基于 Claude Code 构建的 AI 求职框架:评估职位、定制简历、撰写求职信、准备面试。Fork 它并拥有它。The job search that runs on your machine. AI job application framework built on Claude Code: evaluate postings, tailor CVs, write cover letters, prep interviews. Fork it and own it.
Claude Code 的学术研究 Skills:research → write → review → revise → finalizeAcademic Research Skills for Claude Code: research → write → review → revise → finalize
面向 Claude Code 和 Codex 的 AI 视频 skill —— 基于 Remotion 制作电影级产品视频:含 152 张分镜配方卡、209 个动效预览,以及一套开箱即用的模板。AI video skill for Claude Code & Codex — cinematic product videos with Remotion: 152 shot recipe cards, 209 motion previews, a production-ready template