Learning strictly orthogonal p-order nonnegative Laplacian embedding via smoothed iterative reweighted method

Abstract

Laplacian embedding is a powerful graph based method with its ability in spectral clustering to reveal the intrinsic geometry of data in the high dimensional space. Imposing the orthogonality and the nonnegativity constraints can avoid degenerate and negative solutions, respectively. These two attributes are critical yet challenging to achieve simultaneously. Although, in recent years, many attempts have been made to overcome this, this problem is still not perfectly handled. We propose an effective algorithm to solve the Laplacian embedding problem that satisfies the both constraints. To promote the robustness of our embedding model against outliers, we exploit the p-order of the l2-norm distances to find the best solution of the spectral embedding from the input graph. Optimization with both orthonormal and nonnegative constraints is highly nonlinear and nonconvex in feasible domain. The p-order term in our objective further makes it nonsmooth and difficult to efficiently solve in general. We introduce a novel smoothed iterative reweighted method with a smoothness term to tackle this challenging optimization problem and rigorously analyze its convergence. We demonstrate the effectiveness and potential of our proposed method by extensive empirical studies on both synthetic and real data sets.