Learning from Representation Geometry: Toward Robust Supervised and Unsupervised Learning

QC 20260924

Abstract

The rapid advances in data collection and computational power have significantly increased the adoption of machine learning (ML) models. This surge in ML models is mainly due to their ability to generalize across distributions of discriminative features. To achieve this, ML models are trained to parameterize discriminative features by projecting input data into embedding spaces. This mapping imposes a latent geometric structure where feature similarity manifests as spatial clustering, while dissimilarity results in geometric separation. While classification leverages discriminative features to label inputs in supervised settings, unsupervised clustering plays a complementary role by discovering structure when labels are unavailable. However, existing classification approaches cannot recognize classes absent from their training data, which severely degrades their performance upon deployment. Additionally, current clustering methods are not entirely unsupervised, as they still require ground-truth supervision to optimize key hyperparameters for competitive performance. This thesis addresses these limitations that arise from the geometry of learned representations: (i)detecting out-of-distribution (OOD) data in supervised settings, and (ii) discovering structure in unlabeled data via unsupervised spectral clustering. Conventional classifiers often fail when applied to unseen classes, whereas kernel-based spectral clustering is sensitive to manually chosen parameters. To address these limitations, the thesis develops principled methods: calibrated uncertainty estimation for reliable OOD detection, including ensembles that capture temporal coherence in time-series data; an empirical characterization of the geometry of ID and OOD embeddings; a threshold-free detection approach using synthetic OOD embeddings generated from ID data; and a fully unsupervised kernel spectral clustering method where kernel parameters are optimized via a convex objective. A unifying geometric perspective connects these contributions, showing how representation spaces can be exploited to separate in- from out-of-distribution inputs and to discover structure without labels. These contributions are validated empirically: studies across architectures and datasets demonstrate that the proposed OOD detector is model- and data-agnostic, and that our spectral clustering procedure reliably selects effective kernel parameters, improving unsupervised partition quality without labels.

Link to DiVA