Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DAE-Net: Deforming Auto-Encoder for fine-grained shape co-segmentation (2311.13125v2)

Published 22 Nov 2023 in cs.CV and cs.GR

Abstract: We present an unsupervised 3D shape co-segmentation method which learns a set of deformable part templates from a shape collection. To accommodate structural variations in the collection, our network composes each shape by a selected subset of template parts which are affine-transformed. To maximize the expressive power of the part templates, we introduce a per-part deformation network to enable the modeling of diverse parts with substantial geometry variations, while imposing constraints on the deformation capacity to ensure fidelity to the originally represented parts. We also propose a training scheme to effectively overcome local minima. Architecturally, our network is a branched autoencoder, with a CNN encoder taking a voxel shape as input and producing per-part transformation matrices, latent codes, and part existence scores, and the decoder outputting point occupancies to define the reconstruction loss. Our network, coined DAE-Net for Deforming Auto-Encoder, can achieve unsupervised 3D shape co-segmentation that yields fine-grained, compact, and meaningful parts that are consistent across diverse shapes. We conduct extensive experiments on the ShapeNet Part dataset, DFAUST, and an animal subset of Objaverse to show superior performance over prior methods. Code and data are available at https://github.com/czq142857/DAE-Net.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (78)
  1. Zero-shot 3D shape correspondence. In SIGGRAPH Asia 2023 Conference Papers. 1–11.
  2. SATR: Zero-shot semantic segmentation of 3D Shapes. In ICCV.
  3. Jean-Baptiste Alayrac et al. 2022. Flamingo: a visual language model for few-shot learning. NeurIPS (2022).
  4. Dynamic FAUST: Registering Human Bodies in Motion. In CVPR.
  5. Zero-shot semantic segmentation. NeurIPS (2019).
  6. Emerging properties in self-supervised vision transformers. In CVPR.
  7. ShapeNet: An Information-Rich 3D Model Repository. arXiv (2015).
  8. Learning Generative Models of 3D Structures. Computer Graphics Forum (Eurographics STAR) (2020).
  9. Video object cosegmentation. In Proceedings of the 20th ACM international conference on Multimedia.
  10. Zero-shot point cloud segmentation by transferring geometric primitives. arXiv (2022).
  11. DECOR-GAN: 3D Shape Detailization by Conditional Refinement. CVPR (2021).
  12. BSP-Net: Generating Compact Meshes via Binary Space Partitioning. CVPR (2020).
  13. BAE-NET: Branched Autoencoder for Shape Co-Segmentation. ICCV (2019).
  14. Zhiqin Chen and Hao Zhang. 2019. Learning Implicit Fields for Generative Shape Modeling. CVPR (2019).
  15. 3D Highlighter: Localizing regions on 3D shapes via text descriptions. In CVPR.
  16. Objaverse: A universe of annotated 3d objects. In CVPR.
  17. CvxNet: Learnable convex decomposition. In CVPR.
  18. Deformed implicit field: Modeling 3d shapes with learned dense correspondence. In CVPR. 10286–10296.
  19. Learning elementary structures for 3D shape generation and matching. NeurIPS (2019).
  20. PLA: Language-driven open-vocabulary 3D scene understanding. In CVPR.
  21. NeRF-SOS: Any-view self-supervised object segmentation on complex scenes. In ICLR.
  22. Panoptic NeRF: 3D-to-2D label transfer for panoptic urban scene segmentation. In 3DV.
  23. Interactive segmentation of radiance fields. In CVPR.
  24. A. Golovinskiy and T. Funkhouser. 2009a. Consistent segmentation of 3D models. Computers & Graphics (Proc. of SMI) 33, 3 (2009), 262–269.
  25. Aleksey Golovinskiy and Thomas Funkhouser. 2009b. Consistent segmentation of 3D models. Computers & Graphics (2009).
  26. Transforming auto-encoders. In International Conference on Artificial Neural Networks.
  27. 3D concept learning and reasoning from multi-view images. In CVPR.
  28. Co-segmentation of 3D shapes via subspace clustering. In Comput. Graph. Forum.
  29. Qixing Huang and Leonidas Guibas. 2013. Consistent shape maps via semidefinite programming. Computer Graphics Forum (SGP) 32, 5 (2013), 177–186.
  30. Joint shape segmentation with linear programming. ACM TOG (2011).
  31. Learning Shape Primitives via Implicit Convexity Regularization. In ICCV. 3642–3651.
  32. Scaling up visual and vision-language representation learning with noisy text supervision. In ICML.
  33. DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. In CVPR.
  34. Neural star domain as primitive representation. NeurIPS (2020).
  35. Semantic-Aware Implicit Template Learning via Part Deformation Consistency. In ICCV. 593–603.
  36. Diederik P. Kingma and Jimmy Ba. 2015. Adam: a method for stochastic optimization. In ICLR.
  37. Decomposing NeRF for editing via feature field distillation. NeurIPS (2022).
  38. PartGlot: Learning shape part segmentation from language reference games. In CVPR.
  39. Panoptic neural fields: A semantic object-aware neural scene representation. In CVPR.
  40. Language-driven Semantic Segmentation. In ICLR.
  41. Grounded language-image pre-training. In CVPR.
  42. PartSLIP: Low-shot part segmentation for 3D point clouds via pretrained image-language models. In CVPR.
  43. Neural volumes: learning dynamic renderable volumes from images. ACM TOG (2019).
  44. Unsupervised co-segmentation for 3D shapes using iterative multi-label optimization. CAD (2013).
  45. Occupancy Networks: Learning 3D Reconstruction in Function Space. In CVPR.
  46. Generative zero-shot learning for semantic segmentation of 3D point clouds. In 3DV.
  47. NeRF: Representing scenes as neural radiance fields for view synthesis. In ECCV.
  48. Structure-aware shape processing. In SIGGRAPH Asia Course.
  49. RIM-Net: Recursive implicit fields for unsupervised learning of hierarchical shape structures. In CVPR.
  50. Learning unsupervised hierarchical part decomposition of 3D objects from a single rgb image. In CVPR.
  51. Neural Parts: Learning expressive 3D shape abstractions with invertible neural networks. In CVPR.
  52. Superquadrics revisited: Learning 3D shape parsing beyond cuboids. In CVPR.
  53. Openscene: 3d scene understanding with open vocabularies. In CVPR. 815–824.
  54. Learning transferable visual models from natural language supervision. In ICML.
  55. Dynamic routing between capsules. NeurIPS (2017).
  56. Photorealistic text-to-image diffusion models with deep language understanding. NeurIPS (2022).
  57. Ariel Shamir. 2008. A survey on mesh segmentation techniques. In Comput. Graph. Forum.
  58. Unsupervised 3D shape segmentation and co-segmentation via deep learning. CAGD (2016).
  59. DPF-Net: Combining Explicit Shape Priors in Deformable Primitive Field for Unsupervised Structural Reconstruction of 3D Objects. In ICCV. 14321–14329.
  60. Panoptic lifting for 3D scene understanding with neural fields. In CVPR.
  61. Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. ACM TOG (2011).
  62. Learning adaptive hierarchical cuboid abstractions of 3D shape collections. ACM TOG (2019).
  63. Generating Part-Aware Editable 3D Shapes Without 3D Supervision. In CVPR.
  64. Neural Feature Fusion Fields: 3D distillation of self-supervised 2D image representations. In 3DV.
  65. Learning shape abstractions by assembling volumetric primitives. In CVPR.
  66. Co-Hierarchical Analysis of Shape Structures. ACM TOG 32, 4 (2013), Article 69.
  67. Object cosegmentation. In CVPR. 2217–2224.
  68. NeSF: Neural semantic fields for generalizable semantic segmentation of 3D scenes. arXiv (2021).
  69. Data-Driven Shape Analysis and Processing. In SIGGRAPH Asia Course.
  70. Style-content separation by anisotropic part scales. ACM TOG (2010).
  71. Unsupervised point cloud object co-segmentation by co-contrastive learning and mutual attention sampling. In ICCV.
  72. An MIL-derived transformer for weakly supervised point cloud segmentation. In CVPR.
  73. Kaizhi Yang and Xuejin Chen. 2021. Unsupervised learning for cuboid shape abstraction via joint segmentation from point clouds. ACM Transactions on Graphics (TOG) 40, 4 (2021), 1–11.
  74. A scalable active framework for region annotation in 3D shape collections. ACM TOG (2016).
  75. GLIPv2: Unifying localization and vision-language understanding. NeurIPS (2022).
  76. Deep implicit templates for 3d shape representation. In CVPR. 1429–1439.
  77. In-place scene labelling and understanding with implicit scene representation. In ICCV.
  78. AdaCoSeg: Adaptive shape co-segmentation with group consistency loss. In CVPR.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Zhiqin Chen (21 papers)
  2. Qimin Chen (7 papers)
  3. Hang Zhou (166 papers)
  4. Hao Zhang (948 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets