A comparison of restricted maximum likelihood and method of moments variance estimation for small-sample split-plot experiments

Abstract

Researchers may choose to perform an experiment using a split-plot design over more simple designs such as a completely randomized design or a randomized complete block design in order to conserve scarce resources. A split-plot becomes attractive when some treatment factors are more costly to apply to the experimental units or when it is difficult to change one factor from level to level. In such a case it may be more efficient to apply these costly treatments to a small set of larger experimental units (i.e. whole plots) and then apply the less costly treatments to more numerous smaller experimental units (i.e. subplots) nested within the larger ones. Because the subplot and whole-plot experimental units each have a corresponding variance component, the analysis of a split-plot study is more complicated. Making the split-plot analysis even more challenging, cost considerations may also lead to relatively small sample sizes for the whole-plot treatments. An unintended consequence is that some variance components in the split-plot design’s model may be poorly estimated which in turn may have an unanticipated effect on the type I error rates for tests of the fixed effects. As a motivating example, alfalfa yield data from a field study with a split-plot design with four randomized complete blocks at the whole-plot level serves as the basis for a simulation study to estimate the type I error rates of three fixed effects (whole-plot main effects, subplot main effects and whole-plot by subplot interaction). Several other scenarios where the number of blocks and the relative magnitudes of the variance components are varied are also explored. For each scenario, 10,000 data sets were randomly generated assuming normally distributed errors. Two linear mixed models were fit to each data set using the MIXED procedure in SAS; one method estimates the variance components via restricted maximum likelihood (REML) and the other by the method of moments (MoM) based on the type III sums of squares. The REML models yielded inconsistent type I error rates for some tests of fixed effects compared to the MoM models but improved as the number of blocks increased. MoM models tended to hold their nominal type I error rates to within expected Monte Carlo error.