Deep vision with generative adversarial networks to augment and classify tackle images in American youth football

Abstract

This report presents an application of convolutional neural networks (also known as convnets or CNNs) to the video analysis task of detecting risky tackles in American football via classification of image sequences. The solution approach focuses on fine-tuning of pre-trained convnets, extraction of spatial features, and using generative adversarial networks for data augmentation. American adolescents compete in youth football, one of the riskiest sports in the US with a large proportion of head injuries like concussions, as reported in the Youth Football Surveillance System. To provide a football team and coaches with more convenient and efficient training, deep learning automatically classifies tackle videos that record tackle actions. Inflated 3D Convents (I3D) is used for this task; However, I3D does not have ideal performance when the video data was used to train the model because we lack sufficient data and the label system is complex. Generative adversarial networks (GANs) can efficiently augment data. In this study, the style-based generator, styleGAN, was used to solve data problems. At the same time, three other GAN models were used on the same data set to horizontally compare styleGAN’s performance to the performance of other GAN models. In the end, styleGAN performed best. Although the training data took longer with this model, the results were clearer with a higher resolution showing more player detail. The images generated by styleGAN were more varied than images from other models.