A simulation evaluation of backward elimination and stepwise variable selection in regression analysis

Abstract

A first step in model building in regression analysis often consists of selecting a parsimonious set of independent variables from a pool of candidate independent variables. This report uses simulation to study and compare the performance of two widely used sequential, variable selection algorithms, stepwise and backward elimination. A score is developed to assess the ability of any variable selection method to terminate with the correct model. It is found that backward elimination performs slightly better than stepwise, increasing sample size leads to a relatively small improvement in both methods and that the magnitude of the variance of the error term is the major factor determining the performance of both.