3D Menagerie: Modeling the 3D shape and pose of animals
(1611.07700)Abstract
There has been significant work on learning realistic, articulated, 3D models of the human body. In contrast, there are few such models of animals, despite many applications. The main challenge is that animals are much less cooperative than humans. The best human body models are learned from thousands of 3D scans of people in specific poses, which is infeasible with live animals. Consequently, we learn our model from a small set of 3D scans of toy figurines in arbitrary poses. We employ a novel part-based shape model to compute an initial registration to the scans. We then normalize their pose, learn a statistical shape model, and refine the registrations and the model together. In this way, we accurately align animal scans from different quadruped families with very different shapes and poses. With the registration to a common template we learn a shape space representing animals including lions, cats, dogs, horses, cows and hippos. Animal shapes can be sampled from the model, posed, animated, and fit to data. We demonstrate generalization by fitting it to images of real animals including species not seen in training.
Overview
-
Researchers developed a method for creating 3D models of mammals using a small dataset of toy animal scans.
-
A statistical shape model was derived to capture diverse animal shapes and align scans from various poses.
-
Refinements included a part-based shape model and As-Rigid-As-Possible constraints for more accurate registrations.
-
The Skinned Multi-Animal Linear (SMAL) model was created for generating new shapes and reposing animal figures.
-
The SMAL model generalized well to real animals and unseen species, with potential for various scientific and entertainment applications.
Overview
The paper presents an innovative approach to creating 3D models of four-legged mammals. Unlike existing human body models that utilize large datasets of 3D scans, animal modeling faces unique challenges due to their diversity and the impracticality of acquiring numerous 3D scans. To overcome this, researchers used a small dataset of 3D scans of toy animal figurines and applied techniques from human shape and pose modeling.
Model Development
The first step involved scanning a limited selection of animal figurines and using these scans to compute initial registrations. The researchers normalized the poses of these registrations and then learned a statistical shape model capturing the animals' shape variations. This model successfully aligned scans from different poses and produced a low-dimensional Euclidean shape space representing various animals including cats, dogs, horses, and hippos.
Model Refinement
Further refinements improved initial registrations through a novel part-based shape model and a model-free refinement using As-Rigid-As-Possible constraints. They then applied a pose normalization technique to these refined registrations and derived a shape space via principal component analysis. The result was a Skinned Multi-Animal Linear (SMAL) model that allowed for generation of new shapes and reposing.
Generalization and Applications
The SMAL model showed impressive generalization capabilities. Despite being trained on toy figurine scans, it was able to produce realistic fitting results to images of real animals and even to animals not seen during training. This has potential applications in biology, neuroscience, and entertainment, providing a rich basis for future research and development in animal shape and motion capture.
Conclusion
The creation of realistic 3D animal models from minimal data represents a significant advancement, showing that the strategies learned from human pose and shape modeling can be extended to animals. The paper's approach demonstrates a method to capture the variability in animal shapes across different species, offering promising directions for integrating 3D scans and image data to develop more comprehensive models.
Create an account to read this summary for free: