State representation learning for reinforcement learning setting

Abstract

Most of the Reinforcement Learning(RL) tasks in today's world involve high dimensional data. Training deep reinforcement learning based models is already a very challenging task and varies lot with the choice of hyper parameters, hence learning a good model with these high dimensional data becomes even a tougher job. But if we are able to extract meaningful and descriptive low dimensional features from this data, then the problem of training becomes easier than before because now the agent only needs to learn the underlying policy from the meaningful representation of state instead of also learning to extract meaningful information from the high dimensional state. This differs from the general problem of representation learning in the sense that the features extracted by state representation learning in reinforcement learning setting should not just be a good representation of the high dimensional input, but also evolve with time and be affected by the actions chosen by the agent. This research aims to investigate into this field from the lens of Connected Automated Vehicles (CAVs), starting with a literature review of the existing deep RL methods for CAVs, and implementing them with novel ideas and changes to give a good state representation in the CAV-RL setting. Current course of the study involves the evaluation using Sumo and Flow [14] and is focused on changing local view size. Local view is part of the environment that is visible to our ego vehicle. Changing local view sizeis an important step to generalization of this control problem.