论文

探索以交错方式使用 LLM 同时生成推理轨迹和任务特定动作，使两者产生更大协同：推理轨迹帮助模型归纳、跟踪和更新动作计划以及处理异常，而动作使其与外部源交互以获取额外信息。The use of LLMs are explored to generate both reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two: reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources to gather additional information.

论文表明，使用 Stanford Natural Language Inference 数据集有监督训练的通用句子表示，在广泛的迁移任务上能持续优于 SkipThought vectors 等无监督方法。It is shown how universal sentence representations trained using the supervised data of the Stanford Natural Language Inference datasets can consistently outperform unsupervised methods like SkipThought vectors on a wide range of transfer tasks.

本文提出一种权重裁剪的替代方案：对 critic 相对于其输入的梯度范数施加惩罚。其性能优于标准 WGAN，能以几乎无需调参的方式稳定训练多种 GAN 架构。This work proposes an alternative to clipping weights: penalize the norm of gradient of the critic with respect to its input, which performs better than standard WGAN and enables stable training of a wide variety of GAN architectures with almost no hyperparameter tuning.

当将监督规模扩展到 680,000 小时的多语言、多任务数据时，所得到的模型在标准 benchmark 上泛化良好，在 zero-shot transfer 设置下常可与此前全监督方法的结果相当，且无需任何微调。When scaled to 680,000 hours of multilingual and multitask supervision, the resulting models generalize well to standard benchmarks and are often competitive with prior fully supervised results but in a zero-shot transfer setting without the need for any fine-tuning.