Filter-based reinforcement learning for adaptive optimal control of continuous-time dynamical systems

Abstract

The Report describes the Linear Quadratic Regulator(LQR) and the study of the linear systems on different types of systems i.e single agent and multi-agent systems.Different approaches are available for creating an optimal controller for the Linear system where the dynamics of the systems are unknown. One such approach is approximate/adaptive dynamic programming approach(ADP). The algorithm have been implemented for both single and multi-agent system. The algorithm have two level of working one is off-policy data storage and rank checking and the second level is on-policy online reinforcement iterative approach.This paper follows a similar techniques as defined in [1] using dual layer filtering to achieve the optimal control values. In the first layer we are eliminating the state derivative and in the second layer we are iterating under the initial excitation technique to get the optimal values for our dynamical system. The Initial Excitation described in this paper method solve the problem of data storage in the Adaptive Optimal Control(AOC) method and uses a two-level filter approach. The algorithm is designed for single agent system and it was implemented on multi-agent system.