Compressive system identification (CSI): theory and applications of exploiting sparsity in the analysis of high-dimensional dynamical systems

Abstract

The information content of many phenomena of practical interest is often much less than what is suggested by their actual size. As an inspiring example, one active research area in biology is to understand the relations between the genes. While the number of genes in a so-called gene network can be large, the number of contributing genes to each given gene in the network is usually small compared to the size of the network. In other words, the behavior of each gene can be expressed as a sparse combination of other genes. The purpose of this thesis is to develop new theory and algorithms for exploiting this type of simplicity in the analysis of high-dimensional dynamical systems with a particular focus on system identification and estimation. In particular, we consider systems with a high-dimensional but sparse impulse response, large-scale interconnected dynamical systems when the associated graph has a sparse flow, linear time-varying systems with few piecewise-constant parameter changes, and systems with a high-dimensional but sparse initial state. We categorize all of these problems under the common theme of Compressive System Identification (CSI) in which one aims at identifying some facts (e.g., the impulse response of the system, the underlying topology of the interconnected graph, or the initial state of the system) about the system under study from the smallest possible number of observations. Our work is inspired by the field of Compressive Sensing (CS) which is a recent paradigm in signal processing for sparse signal recovery. The CS recovery problem states that a sparse signal can be recovered from a small number of random linear measurements. Compared to the standard CS setup, however, we deal with structured sparse signals (e.g., block-sparse signals) and structured measurement matrices (e.g., Toeplitz matrices) where the structure is implied by the system under study.