World Models Beyond Autoregressive Next State Prediction

Learned models of system dynamics provide an appealing way of predicting the future outcomes in a system, enabling downstream usage for planning or off-policy evaluation in applications such as robotics. However, the prevalent paradigm of autoregressive, next-state prediction in learning dynamics models is challenging to scale to environments with high dimensional observations and long horizons. In this talk, I will present alternative techniques for model learning that go beyond directly predicting next states. Firstly, we will discuss a reconstruction-free class of models that go beyond next-observation prediction by learning the evolution of task-directed latent representations for high dimensional observation spaces. We will then show how this can be generalized to learning a new class of models that avoid autoregressive prediction altogether by directly modeling long-term cumulative outcomes, while remaining task agnostic. In doing so, this talk will propose alternative ways of thinking about model learning that retain the benefits of transferability and efficiency from model-based RL, while going beyond next-state prediction.