Optimization and data-driven methods for signal processing

Abstract

By exploiting and leveraging the intrinsic properties of the observed signal, many signal processing and machine learning problems can be effectively solved by transforming them into optimization problems, which constitutes the first part of the thesis. The theoretical sample complexity for exact signal recovery and the recovery error bound with noisy observation can be derived for the optimization methods. However, it is not efficient for optimization methods to deal with high-dimensional signals and observation with the complex noise and non-stationary sensing process. Thus, in the second part of the thesis, we focus on applying data-driven methods using deep learning techniques to high-dimensional problems in order to verify and examine their efficiency and capability of handling the complex noise and complicated sensing process in real data. Finally, in the third part, we develop optimization-inspired data-driven methods for several inverse problems in signal processing and machine learning. Experiments show that the proposed optimization-inspired data-driven methods can achieve a comparable performance of the optimization methods, are extremely efficient in handling high-dimensional signals, and are very robust against the noise and complicated sensing process. This reveals the potential to design data-driven methods, following traditional optimization approaches, to robustly address challenging problems in signal processing and machine learning. \textit{Part 1: Optimization Methods}. In this part, we apply optimization methods to several inverse problems in signal processing and machine learning, including the signal and support recovery problems for the sparse signal with non-stationary modulation and parameter estimation of damped exponentials. For the inverse problems of sparse signal with non-stationary modulation, we derive the theoretical sufficient sample complexity for exact recovery and bound the signal recovery error in the noisy case. \textit{Part 2: Data-driven Methods}. In this part, we apply data-driven methods to several machine learning problems, which include recognizing the 3-dimensional (3D) chess pieces and classifying and clustering inlier correspondences of multiple objects in computer vision. The experiment results demonstrate the efficiency and robustness of data-driven methods against complex noise in the high-dimensional real data. \textit{Part 3: Optimization-inspired Data-driven Methods}. In this part, we develop data-driven methods based on the optimization techniques. By unfolding the optimization methods and making the parameters trainable, we obtain deep architectures that can achieve a fast approximation of the original optimization approaches and deal with signal models with the complicated sensing process that can not be modeled properly by optimization methods. We also design deep networks following the atomic norm optimization process for multiband signal identification and parameter estimation of contaminated damped exponentials.