Segmentation and tracking of non-planar templates to improve VSLAM

Abstract

In this thesis we address the problem of visual simultaneous localization and mapping (VSLAM) using a single camera as the sole sensor. A VSLAM system estimates its position and orientation (pose) by tracking distinct landmarks in the environment using its camera. Most approaches detect feature points in the environment, using an interest point operator that looks for small textured image templates. Existing algorithms typically assume that an image template is the projection of a single planar surface patch. However, if the template is actually the projection of a nonplanar surface, tracking will eventually fail. We present an algorithm that estimates the 3D structure of a nonplanar template as it is tracked through a sequence of images. Our approach is to model the template as "partially planar", meaning that we assume that the image patch is the projection of a planar surface, but some of the points in the template may not belong to that surface. We automatically identify the points belonging to the "dominant" plane of the patch, and estimate the parameters of that plane. Using this information, the algorithm can more accurately predict the appearance of the template, and as a result, is better able to track the template. We evaluate the benefit of using the new feature tracking method in VSLAM, and demonstrate that the new algorithm can track points longer, and achieve better accuracy, than if the standard singleplane feature tracking method is used. The approach is especially eeffective in scenes where surface discontinuities are common.