Digital humans in physical activity

Abstract

In this work, we build on last semester’s work to improve how we give personalized exercise feedback using 3D human pose data. We start with two methods we already implemented: motion retargeting, which maps a therapist’s movements onto a patient’s body shape using SMPL, and shape-invariant embeddings trained with contrastive learning to focus on motion rather than body differences. This semester, we added two new components. First, a RotJoints alignment pipeline that converts SMPL poses into Euler angles and normalizes them, then ap plies full-sequence Dynamic Time Warping and an improved subsequence DTW for more flexible temporal matching. Second, a lightweight transformer model trained on the MinT dataset to predict muscle activations from SMPL pose sequences. Qualitative examples show that our subsequence alignment yields smoother synchronization compared to the basic DTW approach, and our muscle-activation model produces plausible activation patterns. Finally, we propose combining these kinematic and physiological cues into a single feedback system for rehabilita tion and fitness applications.