Hierarchical Neural Coding for Controllable CAD Model Generation (2307.00149v1)
Abstract: This paper presents a novel generative model for Computer Aided Design (CAD) that 1) represents high-level design concepts of a CAD model as a three-level hierarchical tree of neural codes, from global part arrangement down to local curve geometry; and 2) controls the generation or completion of CAD models by specifying the target design using a code tree. Concretely, a novel variant of a vector quantized VAE with "masked skip connection" extracts design variations as neural codebooks at three levels. Two-stage cascaded auto-regressive transformers learn to generate code trees from incomplete CAD models and then complete CAD models following the intended design. Extensive experiments demonstrate superior performance on conventional tasks such as random generation while enabling novel interaction capabilities on conditional generation tasks. The code is available at https://github.com/samxuxiang/hnc-cad.
- Learning representations and generative models for 3d point clouds. In International conference on machine learning, pp. 40–49. PMLR, 2018.
- Parametric cad modeling: An analysis of strategies for design reusability. Computer-Aided Design, 74:18–31, 2016.
- Bsp-net: Generating compact meshes via binary space partitioning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 45–54, 2020.
- Play: Parametrically conditioned layout generation using latent diffusion. arXiv preprint arXiv:2301.11529, 2023.
- Jukebox: A generative model for music. arXiv preprint arXiv:2005.00341, 2020.
- Inversecsg: Automatic conversion of 3d models to csg trees. Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), 37(6):1–16, 2018.
- How do humans sketch objects? ACM Trans. Graph. (Proc. SIGGRAPH), 31(4):44:1–44:10, 2012.
- Write, execute, assess: Program synthesis with a repl. In Advances in Neural Information Processing Systems (NeurIPS), pp. 9169–9178, 2019.
- Computer-aided design as language. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
- Complexgen: Cad reconstruction by b-rep chain complex generation. ACM Trans. Graph. (SIGGRAPH), 41(4), July 2022. doi: 10.1145/3528223.3530078. URL https://doi.org/10.1145/3528223.3530078.
- Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
- Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000–16009, 2022.
- Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
- The curious case of neural text degeneration. In International Conference on Learning Representations (ICLR), 2020.
- Squared earth mover’s distance-based loss for training deep neural networks. arXiv preprint arXiv:1611.05916, 2016.
- Solidgen: An autoregressive model for direct b-rep synthesis. ”arXiv Preprint”, 2022. doi: 10.48550/ARXIV.2203.13944. URL https://arxiv.org/abs/2203.13944.
- Ucsg-net-unsupervised discovering of constructive solid geometry tree. Advances in Neural Information Processing Systems, 33:8776–8786, 2020.
- Zero-shot cad program re-parameterization for interactive manipulation. arXiv preprint arXiv:2306.03217, 2023.
- Reconstructing editable prismatic cad from rounded voxel models. In SIGGRAPH Asia, December 2022.
- Sketch2cad: Sequential cad modeling by sketching in context. ACM Transactions on Graphics (TOG), 39(6):1–14, 2020.
- Free2cad: Parsing freehand drawings into cad commands. ACM Trans. Graph. (Proceedings of SIGGRAPH 2022), 41(4):93:1–93:16, 2022.
- Secad-net: Self-supervised cad reconstruction by learning sketch-extrude operations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16816–16826, 2023.
- Decoupled weight decay regularization. In International Conference on Learning Representations, 2018.
- Martin, D. What is design intent?, 2023. Accessed: Janurary 19, 2023.
- Mishra, A. Machine learning in the aws cloud: Add intelligence to applications with amazon sagemaker and amazon rekognition, 2019. URL https://aws.amazon.com/sagemaker/groundtruth/.
- Programming language tools and techniques for 3d printing. In 2nd Summit on Advances in Programming Languages (SNAPL 2017). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2017.
- Functional programming for compiling and decompiling computer-aided design. Proceedings of the ACM on Programming Languages, 2(ICFP):1–31, 2018.
- Revisiting the design intent concept in the context of mechanical cad education. Computer-Aided Design and Applications, 15(1):47–60, 2018. doi: 10.1080/16864360.2017.1353733. URL https://doi.org/10.1080/16864360.2017.1353733.
- Sketchgen: Generating constrained cad sketches. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
- Generating diverse high-fidelity images with vq-vae-2. Advances in neural information processing systems, 32, 2019.
- Csg-stump: A learning friendly csg-like representation for interpretable shape parsing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12478–12487, 2021.
- Extrudenet: Unsupervised inverse sketch-and-extrude for shape parsing. In ECCV, 2022.
- SketchGraphs: A large-scale dataset for modeling relational geometry in computer-aided design. In ICML 2020 Workshop on Object-Oriented Learning, 2020.
- Vitruvion: A generative model of parametric cad sketches. arXiv:2109.14124, 2021.
- Shahin, T. Feature-based design – an overview. Computer-aided Design and Applications, 5, 01 2008. doi: 10.3722/cadaps.2008.639-653.
- Csgnet: Neural shape parser for constructive solid geometry. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
- Parsenet: A parametric surface fitting network for 3d point clouds. In European Conference on Computer Vision (ECCV), pp. 261–276. Springer, 2020.
- Learning manifold patch-based representations of man-made shapes. In International Conference on Learning Representations (ICLR), 2021.
- Learning to infer and execute 3d shape programs. In International Conference on Learning Representations (ICLR), 2019.
- Point2cyl: Reverse engineering 3d objects from point clouds to extrusion cylinders. CoRR, abs/2112.09329, 2021. URL https://arxiv.org/abs/2112.09329.
- Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
- Neural face identification in a 2d wireframe projection of a manifold object. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1612–1621, 2022. doi: 10.1109/CVPR52688.2022.00167.
- Pie-net: Parametric inference of point cloud edges. In Advances in Neural Information Processing Systems, volume 33, pp. 20167–20178. Curran Associates, Inc., 2020.
- Engineering sketch generation for computer-aided design. In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshop), pp. 2105–2114, 2021a.
- Fusion 360 gallery: A dataset and environment for programmatic cad construction from human design sequences. ACM Transactions on Graphics (TOG), 40(4), 2021b.
- Deepcad: A deep generative network for computer-aided design models. In IEEE International Conference on Computer Vision (ICCV), pp. 6772–6782, October 2021.
- Iconshop: Text-based vector icon synthesis with autoregressive transformers. arXiv preprint arXiv:2304.14400, 2023.
- Inferring cad modeling sequences using zone graphs. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6062–6070, June 2021.
- Skexgen: Autoregressive generation of cad construction sequences with disentangled codebooks. In International Conference on Machine Learning, pp. 24698–24724. PMLR, 2022.
- Yares, E. The failed promise of parametric cad, 2013. Accessed: November 5, 2022.
- Capri-net: Learning compact cad shapes with adaptive primitive assembly. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11768–11778, 2022.
- Dualcsg: Learning dual csg trees for general and compact cad modeling. arXiv preprint arXiv:2301.11497, 2023.