Bayesian solutions to high-dimensional data challenges using hybrid search

Abstract

In the era of Big Data, variable selection with high-dimensional data has drawn increasing attention. With a large number of predictors, there rises a big challenge for model fitting and prediction. In this dissertation, we propose three different yet interconnected methodologies, which include theory, computation, and real applications for various scenarios of regression analysis. The primary goal in this dissertation is to develop powerful Bayesian solutions to high-dimensional data challenges using a new variable selection strategy, called hybrid search. To effectively reduce computation costs in high-dimensional data analysis, we propose novel computational strategies that can quickly evaluate a large number of marginal likelihoods simultaneously within a single computation. In Chapter 1, we discuss background and current challenges in high-dimensional variable selection. The motivation of our study is also justified. In Chapter 2, we introduce a new Bayesian method of best subset selection in the context of linear regression. The proposed method rapidly finds the best subset via a hybrid search algorithm that combines deterministic local search and stochastic global search. In Chapter 3, on the basis of the approach in Chapter 2, we extend it to a framework of multivariate linear regression model, which analyzes the relationship between multiple response variables and a common set of predictors. In Chapter 4, we propose a general Bayesian method to perform high-dimensional variable selection for various data types, such as binary, count, continuous and time-to-event (survival) data. Using Bayesian approximation techniques, we develop a general computing strategy that enables us to assess the marginal likelihoods of many candidate models within a single computation. In addition, to accelerate the convergence, we employ a hybrid search algorithm that can quickly explore the model spaces and accurately obtain the global maximum of marginal posterior probabilities.