Sparse Bayesian inference using reduced-rank regression approaches

Abstract

In multivariate regression analysis, reduced-rank regression (RRR) has received considerable attention as a powerful way of improving estimation and prediction performances. In this dissertation, we aim to address challenges of dimension reduction associated with rank selection and variable selection in RRR. Our proposed methods are developed in a Bayesian framework so that the uncertainties of rank selection and variable selection can be integrated out via marginalization. We pay special attention to high-dimensional problems in which the number of potential predictors is greater than the sample size. We propose new posterior computation schemes to tackle high-dimensional data challenges under the RRR framework. A great merit of our proposed methods is that they are applicable to a variety of statistical models and machine learning methods including generalized linear models and support vector machines. In addition, various posterior sampling strategies are proposed for handling a variety of rank selection and variable selection problems. To investigate the performance of our proposed methods, simulation study and real data analysis are extensively implemented. This dissertation consists of five chapters. In Chapter 1, we discuss the background and motivation of our study. In Chapter 2, we develop a fully Bayesian approach for high-dimensional RRR problems. In Chapter 3, we propose a multivariate extension of generalized linear models using the sparse RRR idea to handle various data types, including binary, count, and continuous responses. In Chapter 4, we develop a new support vector machine approach for multivariate binary outcomes by incorporating the RRR scheme into the Bayesian support vector machine framework. In Chapter 5, we discuss some remarks and future directions.