Estimating the average treatment effect using the cluster hierarchy and merge post-stratification method

Abstract

Randomized experiments help reduce bias in estimates of the average treatment effect by ensuring that confounders have the same distribution across treatment groups. However, some randomizations can still have imbalances on important confounders, which can lead to inaccurate estimates. Post-stratification is one method for correcting these imbalances to improve estimates. In post-stratification, we form groups of units, called strata, and estimate the overall treatment effect by taking a weighted average of treatment effects within each stratum. In practice, strata are formed based on the values of the confounders. We examine the ad-hoc post-stratification method, where we form groups of units so that every group has at least one treated and control unit. A sufficient condition for the unbiasedness of post-stratification estimators is treatment assignment symmetry—that conditioned on the number of treated units within each stratum, each treatment assignment is equally likely. However, ensuring that each stratum has at least one treatment status often violates assignment symmetry and leads to biased estimates. This report considers a new method for forming strata— cluster hierarchy and merge post-stratification (CHAMP)—that ensures that each treatment status is represented within each stratum and satisfies a weaker form of assignment symmetry required for unbiased estimation. We perform a simulation study to compare CHAMP post-stratification with ad-hoc methods for forming strata. We show that CHAMP post-stratification successfully eliminates bias while ensuring small standard errors of post-stratification estimators. Finally, we apply our method to the Study to Understand Prognoses and Preferences for Outcomes and Risks and Treatments (SUPPORT) dataset to assess the efficacy of right heart catheterization in the initial care of critically ill patients.