Abstract:
This report shows my work where I introduced a unified framework for constructing accurate coresets in machine learning. Accurate coresets are weighted subsets of the original data that preserve the loss function of a model, ensuring that a model trained on the coreset achieves the same performance as one trained on the full dataset. We investigate the impact of lambda values on the size of the coreset, showing that for ridge regression, the coreset size decreases with increasing regularization parameter λ, and for ℓp-regularized ℓp-regression, the coreset size is proportional to o(dp), where d is the data dimensionality and p is the regularization order. Additionally, we extend the coreset construction methods to a wide range of latent variable models, such as Gaussian Mixture Models (GMM), Hidden Markov Models (HMM), and Latent Dirichlet Allocation (LDA), where the coreset size is reduced to O(k3), with k rep resenting the number of latent variables. The proposed algorithms use a Kernelization technique to map data to high-dimensional spaces, preserving model accuracy while reducing computational costs. Our empirical evaluations on real datasets demonstrate significant reductions in coreset size and training time, with less than 1% of the original data required for latent variable models.
