Automatic rigging of animation skeletons with stacked hourglass networks and curve skeleton extraction

Abstract

This thesis presents an approach to the inverse rigging problem that combines ways to extract skeleton poses using deep learning pose estimation networks. This process uses graphical modelling software like Blender and Autodesk Maya with Python library modules that implement mesh contraction algorithms. Mesh contraction processes can produce and extract curve skeletons. Curve skeletons contain properties consistent with general animated rig principles and provide valuable information about medial surfaces, including topological knowledge. They are locally centered, fully connected, and invariant to pose and scale. I extracted curve skeletons from a dataset of model resources and included the curve skeletons as a geometric input feature to a Stacked Hourglass Network. Stacked Hourglass Networks are state-of-the-art architectures that can identify joint and bone locations while learning local detailing within the global context. I adapted a variant Stacked Hourglass network with 3D volumetric input representation and experimented with five combinations of geometric features: topology with signed distance function, topology with signed distance function and (2) principle surface curvatures, topology with signed distance function and local vertex density, topology with signed distance function and local shape diameter, and topology with all geometric features (signed distance function, (2) principle surface curvatures, local vertex density, and local shape diameter). My results show topological information obtained from curve skeletons could be more useful than any combination of these features. I demonstrated and compared my results between ground-truth rigs and network predicted rigs using the average chamfer distance metric.