论文

本文指出基于 Transformer 的次二次时间模型的关键缺陷在于无法执行基于内容的推理，并将选择性 SSM 集成到不包含注意力乃至 MLP 块的简化端到端神经网络架构（Mamba）中。This work identifies that a key weakness of subquadratic-time models based on Transformer architecture is their inability to perform content-based reasoning, and integrates selective SSMs into a simplified end-to-end neural network architecture without attention or even MLP blocks (Mamba).

研究表明，利用变分方法最新进展的深度生成模型与近似贝叶斯推断能够带来显著提升，使生成式方法在半监督学习上极具竞争力。It is shown that deep generative models and approximate Bayesian inference exploiting recent advances in variational methods can be used to provide significant improvements, making generative approaches highly competitive for semi-supervised learning.