Accurate modeling of robot dynamics is essential for model-based control, yet remains challenging under distributional shifts and real-time constraints. In this work, we formulate system identification as an in-context meta-learning problem and benchmark deterministic and generative sequence models for forward dynamics prediction. We take a Transformer-based meta-model, as a strong deterministic baseline, and introduce two complementary diffusion-based approaches: (i) joint trajectory modeling (Diffuser), which learns the full action–state distribution, and (ii) conditioned diffusion models (CNN and Transformer), which generate future states conditioned on control inputs. Through large-scale randomized simulation, we analyze performance across in-distribution and out-of-distribution regimes, as well as computational trade-offs relevant for control. We show that diffusion models significantly improve robustness under distribution shift, with the joint formulation achieving the highest accuracy. Finally, we demonstrate that warm-started sampling enables diffusion models to operate within real-time constraints, making them viable for MPC. These results highlight generative meta-models as a promising direction for robust system identification in robotics.