Continuous time reinforcement learning: A random measure approach

Abstract

We present a random measure approach for modeling exploration, i.e., the execution of measure valued controls, in continuous-time reinforcement learning with controlled diffusion and jumps. We begin with the case when sampling the randomized control in continuous time takes place on a discrete-time grid and reformulate the resulting SDE as an equation driven by suitable random measures. Our main result is a limit theorem for these random measures as the mesh-size of the sampling grid goes to zero. The resulting limit SDE can be applied for the theoretical analysis of exploratory control problems and for the derivation of learning algorithms.