研究库 论文知识库

内容库 / 论文

arXiv:2609.04250 · 多模态

Motion-Omni: End-to-End Joint Speech and Full-Body Motion for Spoken Dialogue

Motion-Omni: End-to-End Joint Speech and Full-Body Motion for Spoken Dialogue

类型：arxiv
标识：2609.04250
链接：https://arxiv.org/abs/2609.04250
主分类：multimodal
形态：method
TLDR：An avatar that holds a conversation should decide what to say and to move while saying it, yet these abilities live in separate model families: spoken dialogue models produce speech without motion, and co-speech motion models produce motion only from audio handed to them. The standard remedy is a cascade that first generates the spoken response and then runs a motion model over the finished audio, which requires a second full inference pass and precludes any joint optimisation between the two. We present Motion-Omni, an end-to-end framework in which a spoken dialogue model natively outputs exp
待LLM分类：否
来源文件：
/inbox/tom/_candidates/2026-09-07-agent-rag-longcontext-candidates.json

相关 · 按语义匹配

笔记精读与批判 · Nemotron-Labs-Audex-30B-A3B · NVIDIA 统一音频-文本 LLM

笔记 MAVIN · 多镜头音视频联合生成 · 批判性精读

文档 Stephen 自测 · R61 · 2026-08-31

笔记 Cool Papers · cs.CL (papers.cool) · RSS 摘要

仓库 suno-ai/bark

笔记 SwanTale · 统一多说话人语音与音频生成 · 轻量精读与批判

arXiv

2609.04250

TLDR

An avatar that holds a conversation should decide what to say and to move while saying it, yet these abilities live in separate model families: spoken dialogue models produce speech without motion, and co-speech motion models produce motion only from audio handed to them. The standard remedy is a cascade that first generates the spoken response and then runs a motion model over the finished audio, which requires a second full inference pass and precludes any joint optimisation between the two. We present Motion-Omni, an end-to-end framework in which a spoken dialogue model natively outputs exp

arXiv ↗ 中文全文