Human activity recognition using MEMS-based motion sensors

Abstract

Micro-electro-mechanical systems (MEMS) are miniature devices that integrate mechanical and electrical components on a single silicon chip, often including sensors and actuators. With sizes ranging from a few micrometers to millimeters, MEMS are diverse in design and functionality. They are pivotal in modern technology and are known for their precision, efficiency, and cost-effectiveness in mass production. They are crucial in advancing consumer electronics, medical devices, and automotive systems. MEMS motion sensors, such as accelerometers and gyroscopes, are essential for capturing detailed time series measurements that reflect dynamic environmental interactions. The thesis explores the utilization of MEMS motion sensors for human activity recognition. We must address many questions to make sense of the motion sensor data. These concern sampling rate, size of data segments, feature extraction, fusing data from multiple sensors, and selection of classification techniques. We focus on finding the optimal sensor sampling rate by balancing the need for detailed data against resource constraints. Our analysis contrasts higher rates, which offer increased detail, against lower rates that are advantageous for their power-saving features. The research involves refining the windowing process to segment time-series sensor data into frames effectively, ensuring precise motion capture without losing context, and enhancing the accuracy of ground truth tagging for dependable data in model training and validation. A vital component is feature extraction, where significant information is extracted from sensor data, coupled with identifying complex features through signal processing, to represent diverse human movements accurately. The study examines using MEMS motion sensors as standalone wearables and in combination with environmental and object-embedded sensors. This approach allows for a thorough analysis of human motions, ranging from extensive activities to subtle micro-movements. We comprehensively evaluate various computational methods, from basic empirical algorithms to sophisticated machine learning and deep learning techniques. This evaluation centers on their effectiveness in interpreting and classifying motion data, considering factors like computational efficiency, adaptability, and accuracy. We apply these approaches to explore the range and intricacy of human movements captured by these sensors. We select applications where the movements vary from mm, cm, to m. The sensors are worn, held in hands, or embedded in the surroundings. For movements in mm, we provide an innovative outlook on the analysis of handwriting micro-events in educational contexts, unlocking possibilities to delve into the subtleties of student interactions and learning dynamics. The sensor is worn on the wrist. For movements in cm, we pioneer a novel technique for personal health management centered on precisely monitoring liquid consumption, a key factor in promoting overall health and supporting informed lifestyle decisions. The sensing involves wrist-worn and object-embedded sensors. For movements in mm, we propose a perceptive strategy to tackle the complexities of indoor parking in urban transport, offering an integrated solution that cleverly utilizes everyday devices and environmental data. The sensing is done through a combination of body-mounted and environmental sensors. Together, these studies demonstrate how integrating diverse sensors and computational strategies can provide a far-reaching impact of this research in harnessing MEMS technology for practical everyday applications. They underscore the complexities inherent in accurately capturing human motion across diverse scenarios and the innovative approaches to navigating and overcoming these challenges.