Human activity recognition and gymnastics analysis through depth imagery

Abstract

Depth imagery is transforming many areas of computer vision, such as object recognition, human detection, human activity recognition, and sports analysis. The goal of my work is twofold: (1) use depth imagery to effectively analyze the pommel horse event in men’s gymnastics, and (2) explore and build upon the use of depth imagery to recognize human activities through skeleton representation. I show that my gymnastics analysis system can accurately segment a scene based on depth to identify a ‘depth of interest’, ably recognize activities on the pommel horse using only the gymnast’s silhouette, and provide an informative analysis of the gymnast’s performance. This system runs in real-time on an inexpensive laptop, and has been built into an application in use by elite gymnastics coaches. Furthermore, I present my work expanding on a bio-inspired skeleton representation obtained through depth data. This representation outperforms existing methods in classification accuracy on benchmark datasets. I then show that it can be used to interact in real-time with a Baxter humanoid robot, and is more accurate at recognizing both complete and ongoing interactions than current state-of-the-art methods.