Autoencoders and kalman filters for generative AI

Abstract

With the growing complexity of models like ChatGPT, LLaMA, and other Generative AI mod- els, which demand significant computational resources, applying them in resource-constrained environments becomes a major challenge. This project aims to design a lightweight model for text data using a combination of Autoencoders and Kalman Filters. The goal is to build an efficient model that can perform tasks such as text classification, sentiment analysis, and se- quence modeling without relying on heavy computational resources. This approach will make the model smaller and faster, allowing it to perform well in real-time text processing on devices with limited resources.