Converting data from multi-instance to single-instance representations using p-Order Laplacian projections

Abstract

Fields such as Computer Vision and Natural Language Processing have a high applicability of Machine Learning algorithms. With large amounts of complex data readily available, there are two prominent approaches to handling data complexity using Machine Learning. First, dimensionality reduction methods such as Principal Component Analysis (PCA) or Laplacian Embeddings (LE) can minimize the number of features needed to accurately represent data. This approach is often effective but has two main drawbacks. First, the input to the dimensionality reduction method is a summary of all the components that make up the data and some valuable information may be lost. Second, dimensionality reduction methods are often sensitive to outliers. The second approach to dealing with complex data is Multi-Instance Learning (MIL). MIL introduces a new paradigm for data representation by viewing data as a grouping, called a bag, of instances. Each instance is modeled the same way the whole data would be represented but now the datum is represented as a bag of instances. Multi-Instance representation can be effective since they focus on modeling all the pieces that make up the whole datum. However, in order to use this representation in Machine Learning applications we must use MIL algorithms and cannot directly use traditional Machine Learning algorithms. In this work, we propose a method to tackle the issues that may arise in dimensionality reduction methods and MIL methods. We do this by learning a reduced-dimension, integrated, outlier resilient single instance representation for our data. We first propose a new dimensionality reduction method of p-Order Laplacian Embeddings (pOLE) that is less sensitive to outliers than traditional LE. We then use this method to learn a projection from the instances of each bag in a Multi-Instance representation of data. This projection, combined with the Single-Instance representation of the same data can produce a reduced-dimension, integrated, outlier resilient Single-Instance representation