Unsupervised Clustering for Nuclei Segmentation at Multiple Levels in H&E Histopathology Images

Overview

This project develops and evaluates three independent unsupervised methods for nuclei segmentation in H&E-stained histopathology images — a critical task for digital pathology. Each method leverages a different feature set (spatial, pixel-level, and stain-specific) to perform clustering without the need for labeled data.

Awarded 2nd Prize at the University of Hertfordshire's Data Science Project Club poster presentation for its rigorous comparative analysis and impactful results.

The Problem

Supervised deep learning methods for nuclei analysis depend on large, expert-annotated datasets — a significant bottleneck that limits scalability and slows research. This project explores whether unsupervised clustering frameworks can deliver viable segmentation directly from raw image data.

Approach — Three Methods

All three methods were evaluated on the MoNuSeg 2018 dataset. Each targets a different image representation:

• Method 1 — Superpixel-Level Clustering: Segments the image into superpixels (SLIC) and applies K-Means, GMM, and FCM clustering for coarse tissue region mapping.
• Method 2 — Pixel-Level Clustering: Operates directly on raw pixel color and intensity values to refine localization of nuclear regions.
• Method 3 — Stain-Specific Clustering (Hematoxylin): Uses Macenko normalization to isolate the Hematoxylin stain channel, then clusters solely on this channel — yielding the most accurate nuclei detection.

Key Technologies: Python · Scikit-learn · K-Means · GMM · FCM · SLIC Superpixels · Macenko Stain Normalization · MoNuSeg Dataset

Results

• Method 3 (Stain-Specific) outperformed all others in instance-level nuclei segmentation accuracy.
• Demonstrated that unsupervised clustering is a viable, scalable alternative when labeled data is scarce.
• Recognized with 2nd Prize at the UH Data Science Project Club 2025.
• Provides a reproducible foundation for annotation-free digital pathology pipelines.