Diffeomorphic Mesh Deformation via Efficient Optimal Transport for Cortical Surface Reconstruction
Abstract: Mesh deformation plays a pivotal role in many 3D vision tasks including dynamic simulations, rendering, and reconstruction. However, defining an efficient discrepancy between predicted and target meshes remains an open problem. A prevalent approach in current deep learning is the set-based approach which measures the discrepancy between two surfaces by comparing two randomly sampled point-clouds from the two meshes with Chamfer pseudo-distance. Nevertheless, the set-based approach still has limitations such as lacking a theoretical guarantee for choosing the number of points in sampled point-clouds, and the pseudo-metricity and the quadratic complexity of the Chamfer divergence. To address these issues, we propose a novel metric for learning mesh deformation. The metric is defined by sliced Wasserstein distance on meshes represented as probability measures that generalize the set-based approach. By leveraging probability measure space, we gain flexibility in encoding meshes using diverse forms of probability measures, such as continuous, empirical, and discrete measures via varifold representation. After having encoded probability measures, we can compare meshes by using the sliced Wasserstein distance which is an effective optimal transport distance with linear computational complexity and can provide a fast statistical rate for approximating the surface of meshes. To the end, we employ a neural ordinary differential equation (ODE) to deform the input surface into the target shape by modeling the trajectories of the points on the surface. Our experiments on cortical surface reconstruction demonstrate that our approach surpasses other competing methods in multiple datasets and metrics.
- Learning representations and generative models for 3d point clouds. In International conference on machine learning, pp. 40–49. PMLR, 2018.
- Frederick J Almgren. Plateau’s problem: an invitation to varifold geometry, volume 13. American Mathematical Soc., 1966.
- Wasserstein generative adversarial networks. In International Conference on Machine Learning, pp. 214–223, 2017.
- Vincent Arsigny. Processing data in lie groups: An algebraic approach. Application to non-linear registration and diffusion tensor MRI. PhD thesis, Citeseer, 2004.
- John Ashburner. A fast diffeomorphic image registration algorithm. Neuroimage, 38(1):95–113, 2007.
- Voxelmorph: a learning framework for deformable medical image registration. IEEE transactions on medical imaging, 38(8):1788–1800, 2019.
- Parametric correspondence and chamfer matching: Two new techniques for image matching. Technical report, SRI INTERNATIONAL MENLO PARK CA ARTIFICIAL INTELLIGENCE CENTER, 1977.
- On parameter estimation with the Wasserstein distance. Information and Inference: A Journal of the IMA, 8(4):657–676, 2019.
- Vox2cortex: fast explicit reconstruction of cortical surfaces from 3d mri scans with geometric deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20773–20783, 2022.
- Sliced and radon Wasserstein barycenters of measures. Journal of Mathematical Imaging and Vision, 51:22–45, 2015.
- Nicolas Charon. Analysis of geometric and functional shapes with extensions of currents: applications to registration and atlas estimation. PhD thesis, École normale supérieure de Cachan-ENS Cachan, 2013.
- The varifold representation of nonoriented shapes for diffeomorphic registration. SIAM journal on Imaging Sciences, 6(4):2547–2580, 2013.
- Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
- Learning implicit fields for generative shape modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5939–5948, 2019.
- 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VIII 14, pp. 628–644. Springer, 2016.
- Theory of ordinary differential equations. Pure & Applied Mathematics S. McGraw-Hill Education, London, England, March 1984.
- Deepcsr: A 3d deep learning approach for cortical surface reconstruction. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 806–815, 2021.
- Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In Advances in Neural Information Processing Systems, pp. 2292–2300, 2013.
- Unsupervised learning for fast probabilistic diffeomorphic registration. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 729–738. Springer, 2018.
- Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces. Medical image analysis, 57:226–236, 2019.
- Ppf-foldnet: Unsupervised learning of rotation invariant 3d local descriptors. In Proceedings of the European conference on computer vision (ECCV), pp. 602–618, 2018.
- 3d point cloud denoising via deep neural network based local surface estimation. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8553–8557. IEEE, 2019.
- Interpolating between optimal transport and MMD using Sinkhorn divergences. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 2681–2690, 2019.
- Bruce Fischl. Freesurfer. Neuroimage, 62(2):774–781, 2012.
- Diffeomorphic matching of distributions: A new approach for unlabelled point-sets and sub-manifolds matching. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004., volume 2, pp. II–II. IEEE, 2004.
- Large deformation diffeomorphic metric curve mapping. International journal of computer vision, 80:317–336, 2008.
- Statistical inference with regularized optimal transport. arXiv preprint arXiv:2205.04283, 2022.
- A papier-mâché approach to learning 3d surface generation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 216–224, 2018.
- Kunal Gupta. Neural mesh flow: 3d manifold mesh generation via diffeomorphic flows. University of California, San Diego, 2020.
- Diffeomorphic image registration with neural velocity field. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1869–1879, 2023.
- Hybrid neural diffeomorphic flow for shape representation and generation via triplane. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 7707–7717, 2024.
- Hierarchical surface prediction for 3d object reconstruction. In 2017 International Conference on 3D Vision (3DV), pp. 412–420. IEEE, 2017.
- Diffeomorphic registration of discrete geometric distributions. In Mathematics Of Shapes And Applications, pp. 45–74. World Scientific, 2020.
- The alzheimer’s disease neuroimaging initiative (adni): Mri methods. Journal of Magnetic Resonance Imaging: An Official Journal of the International Society for Magnetic Resonance in Medicine, 27(4):685–691, 2008.
- Local implicit grid representations for 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6001–6010, 2020.
- A general framework for curve and surface comparison and registration with oriented varifolds. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3346–3355, 2017.
- Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
- Do neural optimal transport solvers work? a continuous Wasserstein-2 benchmark. Advances in Neural Information Processing Systems, 34:14593–14605, 2021.
- Learning a probabilistic model for diffeomorphic registration. IEEE transactions on medical imaging, 38(9):2165–2176, 2019.
- Integrating efficient optimal transport and functional maps for unsupervised shape correspondence learning. arXiv preprint arXiv:2403.01781, 2024.
- Corticalflow: a diffeomorphic mesh transformer network for cortical surface reconstruction. Advances in Neural Information Processing Systems, 34:29491–29505, 2021.
- Corticalflow: A diffeomorphic mesh deformation module for cortical surface reconstruction. arXiv preprint arXiv:2206.02374, 2022.
- Marching cubes: A high resolution 3d surface construction algorithm. ACM siggraph computer graphics, 21(4):163–169, 1987.
- A Bayesian generative model for surface template estimation. Journal of Biomedical Imaging, 2010:1–14, 2010.
- Pialnn: a fast deep learning framework for cortical pial surface reconstruction. In Machine Learning in Clinical Neuroimaging: 4th International Workshop, MLCN 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27, 2021, Proceedings 4, pp. 73–81. Springer, 2021.
- Cortexode: Learning cortical surface reconstruction by Neural ODEs. IEEE Transactions on Medical Imaging, 2022.
- Reliability of brain volume measurements: a test-retest dataset. Scientific data, 1(1):1–9, 2014.
- Open access series of imaging studies (oasis): cross-sectional mri data in young, middle aged, nondemented, and demented older adults. Journal of cognitive neuroscience, 19(9):1498–1507, 2007.
- Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem. In Advances in Neural Information Processing Systems, 2019.
- Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4460–4470, 2019.
- Energy-based sliced Wasserstein distance. Advances in neural information processing systems, 2023.
- Sliced Wasserstein estimation with control variates. International Conference on Learning Representations, 2024.
- Distributional sliced-Wasserstein and applications to generative modeling. In International Conference on Learning Representations, 2021a.
- Self-attention amortized distributional projection optimization for sliced Wasserstein point-cloud reconstruction. Proceedings of the 40th International Conference on Machine Learning, 2023.
- Quasi-monte carlo for 3d sliced wasserstein. In The Twelfth International Conference on Learning Representations, 2024a.
- Point-set distances for learning representations of 3d point clouds. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021b.
- Temporal 3d shape modeling for video-based cloth-changing person re-identification. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 173–182, 2024b.
- Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3504–3515, 2020.
- Deep mesh reconstruction from single rgb images via topology modification networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9964–9973, 2019.
- Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 165–174, 2019.
- Computational optimal transport: With applications to data science. Foundations and Trends® in Machine Learning, 11(5-6):355–607, 2019.
- Accelerating 3d deep learning with pytorch3d. arXiv:2007.08501, 2020.
- Multidirectional and topography-based dynamic-scale varifold representations with application to matching developing cortical surfaces. NeuroImage, 135:152–162, 2016.
- Currents, flows and diffeomorphisms. Topology, 14(4):319–327, 1975.
- Corticalflow++: Boosting cortical surface reconstruction accuracy, regularity, and interoperability. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part V, pp. 496–505. Springer, 2022.
- Sinkhorn divergences for unbalanced optimal transport. arXiv preprint arXiv:1910.12958, 2019.
- Geometrics: Exploiting geometric structure for graph-encoded objects. arXiv preprint arXiv:1901.11461, 2019.
- Topology-preserving shape reconstruction and registration via neural diffeomorphic flow. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20845–20855, 2022.
- Hybrid-csr: Coupling explicit and implicit shape representation for cortical surface reconstruction. arXiv preprint arXiv:2307.12299, 2023.
- Octree generating networks: Efficient convolutional architectures for high-resolution 3d outputs. In Proceedings of the IEEE international conference on computer vision, pp. 2088–2096, 2017.
- Surface matching via currents. In Information Processing in Medical Imaging: 19th International Conference, IPMI 2005, Glenwood Springs, CO, USA, July 10-15, 2005. Proceedings 19, pp. 381–392. Springer, 2005.
- Cédric Villani. Topics in optimal transportation. Number 58. American Mathematical Soc., 2003.
- Cédric Villani. Optimal transport: old and new, volume 338. Springer, 2009.
- Martin J Wainwright. High-dimensional statistics: A non-asymptotic viewpoint. Cambridge University Press, 2019.
- Pixel2mesh: Generating 3d mesh models from single rgb images. In Proceedings of the European conference on computer vision (ECCV), pp. 52–67, 2018a.
- Adaptive o-cnn: A patch-based deep representation of 3d shapes. ACM Transactions on Graphics (TOG), 37(6):1–11, 2018b.
- 3dn: 3d deformation network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1038–1046, 2019.
- Voxel2mesh: 3d mesh model generation from volumetric data. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part IV 23, pp. 299–308. Springer, 2020.
- Disn: Deep implicit surface network for high-quality single-view 3d reconstruction. Advances in neural information processing systems, 32, 2019.
- Multiview neural surface reconstruction by disentangling geometry and appearance. Advances in Neural Information Processing Systems, 33:2492–2502, 2020.
- Q Zhou. Pymesh—geometry processing library for python. Software available for download at https://github. com/PyMesh/PyMesh, 2019.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.