Propensity score matching approach to intervention evaluation: the case of the crop production intervention in the USDA Ghana poultry project

Abstract

USDA used its Food for Progress mechanism to support two projects aimed at enhancing productivity in Ghana’s poultry industry. The projects were the Ghana Poultry Project (GPP) implemented by ACDI/VOCA and Assisting in the Management of Poultry and Layer Industries by Feed Improvement and Efficiency Strategies (AMPLIFIES), implemented by the American Soybean Association’s WISHH (World Initiative for Soy in Human Health). While GPP focused on enhancing the capacity of poultry farmers, AMPLIFIES’ focus was on improving poultry feed quantity and quality. As part of AMPLIFIES’s efforts, the project invested in increasing maize and soybean production and post-harvest management in Ghana. Maize and soybean account for the majority of poultry feed ingredients. This research sought to evaluate the impact of AMPLIFIES project on the beneficiary maize and soybean farmers. The study was done using secondary data from the final evaluation of the AMPLIFIES project conducted by METSS Research, a research group led by the Department of Agricultural Economics at Kansas State University. The data were collected using a farm-level survey in the three principal maize and soybean-producing regions: Northern, Brong Ahafo, and Ashanti. The results show that the gross margin of maize beneficiaries of the AMPLIFIES was 50% compared to 40% for non-beneficiaries. Similarly, the gross margin for soybean beneficiaries was 56% compared to 55% for non-beneficiaries. While the beneficiaries’ gross margins were both positive, the maize gross margin difference was statistically significant at the 1 percent level and the soybean gross margin difference was not statistically different. The propensity score matching approach was used to evaluate the extent to which AMPLIFIES’s intervention could allocate the differences between the financial performance of beneficiaries and non-beneficiaries. The study explores two treatment effects: Average Treatment Effect (ATE) and Average Treatment Effect on the Treated (ATT). ATE measures the average treatment effect in the entire population while ATT measures the treatment effect on the treated population. Thus, ATT explores the effect of the treatment on beneficiaries and compares it to a “hypothetical” condition of the same being non-beneficiaries. Generating the “hypothetical “group is achieved through the propensity score matching process. The matching of beneficiaries and non-beneficiaries was based on gender, region, and maize and soybean production experience of producers. Using gross margin as the indicator of interest, the results show that the ATE of the AMPLIFIES intervention was 6.6% (p < 0.10) for maize producers and 7.2% (p < 0.10) for soybean producers. However, the average treatment effect on the treated (comparing beneficiaries to a situation where they were not beneficiaries) was negative for maize and positive for soybean. Neither of the ATT statistics was statistically significant, suggesting that the financial performance of beneficiary producers was not different from their state had they not been beneficiaries, but the intervention produced a positive impact on beneficiaries compared to non-beneficiaries across the population.