Semisupervised clustering algorithm combining SUBCLU and constrained clustering for detecting groups in high dimensional datasets

Abstract

High dimensional data poses a challenge to traditional clustering algorithms, where the similarity measures are not meaningful, affecting the quality of the groups. As a result, subspace clustering algorithms have been proposed as an alternative, aiming to find all groups in all spaces of the dataset.

By detecting groups on lower dimensional spaces, each group may belong to different subspaces of the original dataset. Therefore, attributes the user considers of interest may be excluded in some or all groups, decreasing the value of the result for the data analysts.

In this project, a new algorithm is proposed, that combines SUBCLU and the  clustering algorithms by constraint, which allows the users to identify variables as attributes of interest based on prior knowledge of domain, targeting direct group detection toward spaces that include user’s attributes of interest, and thereafter, generating more meaningful groups.