Sparse Bayesian Learning Algorithms Revisited: From Learning Majorizers to Structured Algorithmic Learning Using Neural Networks

Sparse Bayesian Learning (SBL) is one of the most popular sparse signal recovery methods, and various algorithms exist under the SBL paradigm. However, given a performance metric and a sparse recovery problem, it is difficult to know a-priori the best algorithm to choose. This difficulty is in part due to a lack of a unified framework to derive SBL algorithms. In this work, we address this issue by first showing that the most popular SBL algorithms can be derived using the majorization-minimization (MM) principle. Hence, providing convergence guarantees to this class of SBL methods hitherto unknown. Moreover, we show that the two most popular SBL update rules not only fall under the MM framework but are both valid descent steps for a common majorizer, revealing a deeper analytical compatibility between these algorithms. Using this insight and properties from MM theory we expand the class of SBL algorithms, and address the question of finding the best SBL algorithm via data within the MM framework. Second, we go beyond the MM framework by introducing the powerful modeling capabilities of deep learning to further expand the class of SBL algorithms, and aim to learn a superior SBL update rule from data. We propose a novel deep learning architecture that can outperform the classical MM based ones across different sparse recovery problems. Our architecture is designed such that its complexity does not scale with the dimension of the measurement matrix, and hence provides us with a unique opportunity to test the generalization capability across different measurement matrices. In the case of parameterized dictionaries, the invariance to the size of the matrix allows us to train and test the model across different ranges of the parameter. We also showcase our models ability to learn a functional mapping by its zero-shot performance on unseen measurement matrices. Finally, we test our model performance across different number of snapshots, signal-to-noise ratios and sparsity levels.