Evaluating Body Pose Grouping via Clustering Algorithms

Overview

Training supervised models for human action recognition relies on massive amounts of manually labeled data. This project investigates whether unsupervised clustering algorithms can automatically group continuous streams of 3D human movement coordinates into discrete, identifiable poses — effectively automating the data annotation pipeline.

Using Indian Classical Dance (Bharatanatyam) as a ground-truth dataset — due to its strict, predefined geometric postures — we extracted skeletal data, built a custom data-cleaning filter, and evaluated three distinct clustering algorithms.

The Problem — Pose Hallucination

While tools like MediaPipe are highly efficient at extracting 3D landmarks, they exhibit a critical flaw. When the dancer's legs move out of the camera frame, the model continues to guess their location based on torso orientation — resulting in heavily distorted "collapsed skeletons" that corrupt the dataset. Solving this was a prerequisite for any meaningful clustering.

Approach & Methodology

A rigorous preprocessing pipeline was built before any clustering was applied:

• Dual-Stage Heuristic Filter: Custom mathematical logic to automatically discard corrupted frames — a visibility threshold (dropping frames where ankle confidence fell below 65%) combined with a geometric constraint (dropping frames where vertical hip-to-ankle distance was less than 18% of image height).
• Data Refinement: The filter reduced a noisy dataset of 13,474 raw video frames to a high-fidelity clustering-ready subset of 4,789 frames.
• Clustering Evaluation: 99-feature coordinate vectors were scaled and passed through K-Means (Kneedle algorithm determined optimal k=6), Agglomerative Hierarchical Clustering (validated via dendrograms), and DBSCAN (using OPTICS reachability plots).

Key Technologies: Python · MediaPipe · OpenCV · Scikit-learn · K-Means · DBSCAN · Agglomerative Clustering · t-SNE · PCA

Results

• DBSCAN — standout performer: Its density-based nature successfully identified transitional frames between dance moves and isolated them as noise, keeping main pose clusters highly pure.
• Clear spatial separation: t-SNE projection of the 99-feature vectors into 2D confirmed distinct, non-overlapping physical posture groups.
• Validated pipeline: The methodology proves unsupervised clustering can automate the conversion of raw video coordinate streams into discrete, labelable action groups — with direct applications in elderly fall-detection edge devices and similar action-recognition tasks.