Deep learning in seed image classification using domain randomization and convolutional neural networks

Abstract

The objective of this thesis is to study the application of deep learning in seed image classification using convolutional neural networks. The images are captured using an unmanned aerial vehicle as might be used in a plant breeding program. Plant breeding programs extensively monitor the evolution of seed kernels for seed certification, wherein lies the need to appropriately label the seed kernels by type and quality. However, the breeding environments are large where the monitoring of seed kernels can be challenging due to the relatively small size of seed kernels. For this experiment, we use an unpiloted aerial vehicle to capture images at low altitude while being able to access even the remotest areas in the environments. In a small breeding program, they might use fixed cameras instead. A key bottleneck in the labelling of seeds using UAV imagery is drone altitude, i.e., the classification accuracy decreases as drone altitude increases due to lower image detail. Convolutional neural networks are a great tool for multi-class image classification where there is a training dataset that closely represents the different scenarios that the network might encounter during evaluation. However, with the seeds being in a breeder environment coupled with the varying image resolution and clarity, it is challenging to generate a training dataset that covers all possible evaluation scenario. This work addresses the challenge of training data creation using Domain Randomization wherein synthetic image datasets are generated from a meager sample of seed images captured by the bottom camera of an autonomously driven Parrot AR Drone 2.0. Also, the work includes a seed classification framework as a proof-of-concept using the convolutional neural networks of Microsoft's ResNet-100, Oxford's VGG-16, and VGG-19. To enhance the classification accuracy of the framework, an ensemble model is developed resulting in an overall accuracy of 94.6%. The task of classification is performed on five distinct types of seeds, canola, rough rice, sorghum, soy, and wheat. The second use case is the drones are used for aerial surveying where the UAV imagery is used to estimate the area of a plot. The work demonstrates how the area can be calculated using the pixels of an image and the camera specifications of the drone.