探讨共享工作空间上 LLM Agent 委员会的审议过程是否能被捷径和线索(benchmark 所奖励但临床医生会忽略的)所博弈,以及委员会的社会可信度所构成的游戏。It is asked whether committees of language-model agents deliberating on a shared workspace can be gamed by shortcuts, cues a benchmark rewards but a clinician would ignore, and what games a committee is social plausibility.
论文
185 张论文卡片 · 评测集
提出 GRIP(Grounded Reasoning via Information-Restricted Premises),引入容量不对称:decoder 对 query 保持全维度访问,而检索到的证据则通过一个严苛的随机瓶颈,迫使证据通道仅编码 query 中无法获得的残余信息。GRIP (Grounded Reasoning via Information-Restricted Premises), which imposes capacity asymmetry: the decoder keeps full-dimensional access to the query, while retrieved evidence passes through a severe stochastic bottleneck, which forces the evidence channel to encode only the residual information unavailable from the query.
提出 HarnessEval-W,一个 agentified 的评估 pipeline,将 LLM 生态中的 harness 范式引入 world model 基准测试,并在 330 个评估用例上对 18 个代表性 world model 进行了评估。This work introduces HarnessEval-W, an agentified evaluation pipeline that brings the harness paradigm from the LLM ecosystem to world model benchmarking, and applies HarnessEval-W to 18 representative world models over 330 evaluation cases.
语言模型以一种可被报告的形式持有潜在量,并且当任务需要灵活复用该量时,更多该量的信息会以这种形式存在。Language models hold latent quantities in a form they can report on, and more of a quantity is present in that form when the task requires reusing it flexibly when the task requires reusing it flexibly.
认识到多样化的多参考任务共享一组共同的原子操作,本文形式化了四个算子:Anchor、Disentangle、Apply 和 Compose,并构建了 TRACE-Bench,包含约 1,600 个跨 slot 数量 1–8 的评估用例。Recognizing that diverse multi-reference tasks share a common set of atomic operations, this work formalizes four operators: Anchor, Disentangle, Disentangle, Apply, and Compose, and constructs TRACE-Bench, comprising approximately 1,600 evaluation cases across slot counts 1--8.