提出首个 MuseCP 评估框架,涵盖四类音乐 facet,使用细粒度且量身定制的指标来捕捉音乐属性的细微变化,并希望为开发更有效、更可靠、具备强大 MuseCP 能力的音乐编辑策略提供实践指导。The first MuseCP evaluation framework is introduced that covers four categories of music facets with fine-grained and well-tailored metrics to capture nuanced changes in music attributes and hopes it can offer practical guidance for developing more effective and reliable music editing strategies with strong MuseCP capability.
论文
7 张论文卡片 · 评测基准 · 应用落地
现代 LLM Agent 的改进通常依赖于人工修改 prompt、工具或工作流,而围绕模型的可执行支架——harness——在部署后一般被视为固定不变的产物。本文研究一种替代方案:harness 是任务特定的且持续可进化的,每个任务族维护各自的 harness,通过固定的任务注入接缝在不同迭代间热替换,并依据环境反馈进行改写。我们提出分层自改进(HSI),在该框架中,一个冻结的 LLM M 在三层Modern LLM agents are often improved by modifying prompts, tools, or workflows manually, while the executable scaffold surrounding the model---the harness---is typically treated as a fixed artifact after deployment. This work studies an alternative where the harness is task-specific and continuously evolvable: each task family maintains its own harness, which is hot-swapped across iterations through a fixed task-injection seam and rewritten using environment feedback. We introduce Hierarchical Self-Improvement (HSI), a framework in which a single frozen LLM M operates across three hierarchical
应在真实类别分布下使用跨网络评估来判断部署就绪度,而非仅依赖域内准确率。Deployment readiness should be assessed using cross-network evaluation under realistic class distributions, rather than within-domain accuracy alone, to suggest deployment readiness should be assessed using cross-network evaluation under realistic class distributions, rather than within-domain accuracy alone.
分析表明,借助 LLM,美国约 15% 的工作任务可在保持同等质量的前提下显著提速完成,意味着 LLM 驱动的软件将对底层模型经济影响的规模化产生实质性作用。The analysis suggests that, with access to an LLM, about 15% of all worker tasks in the US could be completed significantly faster at the same level of quality, implying that LLM-powered software will have a substantial effect on scaling the economic impacts of the underlying models.
结果表明,通过结合基础 LLM 改进(PaLM 2)、医学领域微调以及包括新颖集成精化方法在内的提示策略,医学问答正快速接近医生水平的表现。Results highlight rapid progress towards physician-level performance in medical question answering by leveraging a combination of base LLM improvements (PaLM 2), medical domain finetuning, and prompting strategies including a novel ensemble refinement approach.
本文提出一种结构化协议,通过对开源 LLM 评估与安全工具的分类驱动分析来自动化 AI 风险缓解,并给出一个可同时适用于开源与商用方案的分类驱动框架。This paper proposes a structured protocol to automate AI risk mitigation through a taxonomy-driven analysis of open-source LLM evaluation and security tools, and presents a taxonomy-driven framework applicable to open-source and proprietary solutions.
本文给出了 LLM routing 的统一形式化,将其刻画为由五个组件构成的序贯决策过程:context 编码器、模型编码器、评分函数、决策规则和学习信号,涵盖单轮、多轮和个性化 routing。This work presents a unified formulation of LLM routing as a sequential decision process characterized by five components: context encoders, model encoders, scoring functions, decision rules, and learning signals, covering single-turn, multi-turn, and personalized routing.