End-to-end development of a mobile application to count the number of grains within a sorghum panicle using on-farm imagery

Abstract

Estimating yield before harvesting can lead to important decisions related to crop management and use inputs. For the sorghum (Sorghum bicolor L.), yield estimation before harvest is laborious and time-consuming, mainly due to the fact that it relies on the panicle (head) being removed, threshed, and the grains being counted manually or machine-assisted. In this scenario, the integration of computer vision, artificial intelligence algorithms, and mobile applications represents a potential alternative to manual counting of grains on an on-farm scale. Consequently, this study aimed to develop an improved method for grain counting and, subsequently, yield estimation prior to harvesting, by combining artificial intelligence techniques with the development of a mobile application. Chapter 2 assesses the development of artificial intelligence techniques for the automated counting of grains in on-farm images captured using a mobile phone. In the aforementioned chapter, a total of 648 sorghum panicle images were benchmarked using a mobile phone in a field setting. Subsequently, the number of grains in 198 of these images was manually counted. Two frameworks were used to train segmentation models from the images: Yolov8 and Detectron2, with an average precision of 89% and 75%, respectively. Later, we tested three density models for the counting task: MCNN, TCNN-Seed, and Sorghum-Net (developed by this study), with the last overperforming the other two models with 17% of Mean Average Precision Error (MAPE). Lastly, a polynomial linear model was used to relate the counting obtained from the density model to the whole panicle, resulting in a MAPE of 17%. In Chapter 3, the best models from Chapter 2 (Yolov8, Sorghum-Net, and the polynomial model) were embedded into a mobile application for utilization in on-farm applications. We developed two branches of the mobile application: one operating locally and the other assisted by a web server. The user can obtain the yield estimation by inputting information on plant population and crop condition. A laboratory and a field validation were performed, yielding a relative root mean square error of 22.4% and 21.4%, respectively. The outcomes from this study can be employed by farmers, crop insurance companies, agronomists, breeders, and researchers to rapidly estimate and forecast sorghum grain yield before harvesting.