Data driven robotics

Abstract

Achieving dexterous and high-precision autonomous manipulation requires moving beyond pre programmed motion sequences toward adaptable, data-driven policies capable of handling envi ronmental variations. This project presents an implementation of Action Chunking Transformers (ACT), a method that enables robotic manipulators to infer appropriate motion strategies from sensory observations by predicting sequences of future actions rather than single-step trajectories. The underlying architecture utilises an encoder-decoder transformer to process visual inputs from multiple cameras alongside proprioceptive data, employing temporal ensembling to ensure smooth and temporally consistent execution. Experimental validation was conducted on an OpenManipulator-X robotic arm , utilizing a leader follower configuration to curate a dataset of 50 demonstration episodes for randomized pick-and place tasks. Although initial deployments highlighted the model’s sensitivity to visual ambiguity, subsequent optimizations in lighting, scene isolation, and visual contrast significantly enhanced policy localization and grasping robustness. Concluding with a perspective on scalability, we propose future integrations with Vision-Language-Action (VLA) models such as Pi0 and the use of depth streams to synthesize virtual camera views, thereby reducing hardware dependencies.