3D Mesh Hollowing: Solid Mesh to Hollow Mesh Conversion Using Multi-View Data and Neural Networks

2. RELATED WORK

3D Generation with 2D Supervision Adapting 2D image priors using NeRF and GAN-based architectures is done by Mildenhall et al. [2020], as well as Chan et al. [2021a]. Techniques like 2D diffusion models, which are very intensive for per-shape optimization, were used by Poole et al. [2022] to optimize 3D shapes. Recent works focused on multi-view generation with 2D models [Long et al., 2023], but suffer from view inconsistency and complex lighting issues.

These rely on 3D native generative models with representations such as point clouds, meshes, or implicit fields [Li et al., 2018; Nash et al., 2020] as well as GANs or autoregressive models [Goodfellow et al., 2014; Sun et al., 2020]. Recent work in this area is moving toward a new class of diffusion models for 3D objects, which are computationally expensive and fragile about fine geometric details [Ho et al., 2020; Chou et al., 2023].

SDS [Poole et al., 2022] and multi-view diffusion models [Liu et al., 2023; Chan et al., 2023] advanced single-view 3D reconstruction. While optimizing 3D shapes with 2D priors with SDS, one found it rather computationally expensive and prone to inconsistency between views. Recent work like Zero123 [Liu et al., 2023] and SyncDreamer [Chan et al., 2023] incorporate multi-view diffusion that was beneficial for consistency while being nonoptimal in terms of geometry.

Direct 3D generative models using point clouds or implicit fields [Luo and Hu, 2021; Liu et al., 2023] are faster in inference but generalize poorly because the datasets are small. For the remedy, Wonder3D uses a cross-domain diffusion model with cross-domain attention and geometry-aware normal fusion to generate high-quality and efficient 3D reconstructions from single images, which works better than prior methods in speed and detail [Long et al., 2023]

3. METHOD

Conversion of solid meshes to hollow meshes is significant in 3D modeling, which has a wide application in manufacturing, product design, medical modeling, and virtual reality. Hollow mesh models significantly provide the test of physical properties, light-weight structure, and material optimization therefore, it is hard to resolve this problem since an operation, by trying to leave both exterior fidelity of the surface and interior geometric details, would undermine one of the factors while trying to retain the other. The traditional methods fail to achieve internally exact void generation together with surface quality, thereby providing only geometrically inconsistent models or computationally expensive solutions.

We start by introducing a method of mesh hollowing using the Craftsman system; it is an integration of advanced representations of geometry and designs to built architectures. In our system, we utilize the application of the Convolutional by employing specific techniques in geometry to reconstruct the internal and external structures.This technique utilizes what is commonly referred to as Flexicubes-a flexible 3D grid representation on one's side to allow for end-to-end optimization for precise modeling of hollow internal structures while maintaining surface quality. In addition, Signed Distance Fields are encoded in the surface geometry, and the Dual Marching Cubes algorithm is utilized to ensure consistent mesh generation.

3.1 Multi-view Diffusion Model

First, we explain the design of the multi-view diffusion model to generate four orthographic view images from a single input image. Instead of training from scratch, which is typically extremely expensive, we initialize the diffusion models using the checkpoint of the CraftsMan, a high-performance diffusion model for multi-view image generation from a single image. The original Craftsman supports the 6 views generation. We reduce it to 4 views by removing two more perspectives (up and down).

3.2 Convolutional Reconstruction Model

now describe the architecture of the CRM in detail. In Figure 4 it is shown that CRM takes as input four orthographic images along with Canonical Coordinate Maps (CCMs) and processes them using a convolutional U-Net to transform the combined inputs into a rolled-out triplane representation. This representation is then reshaped into a standard triplane format. To recover geometric and texture-meaningful information from the triplane features, small MLPs are used to decode into Signed Distance Field values, texture colors, and Flexicubes parameters. Finally, the decoded values are used for the creation of a textured 3D mesh with the use of the dual marching cubes algorithm. Below, a detailed overview of the core components of CRM is provided.

Triplane Representation: The system uses a triplane-based approach to correctly depict complex geometries by first orienting four orthographic views of the object, its front, back, left, and right to allow the outer as well as inner features to come out. Such a combination is further redefined by using Canonical Coordinate Maps (CCMs) for a proper spatial insight where in the model can infer both the outer and inner geometries in association.

The proposed method is trained on Objaverse 3D datasets and precise internal structures supervision. Besides utilizing the pre-trained checkpoint of the Craftsman system and streamlined inference pipeline, the generation of hollow meshes is performed very rapidly, showing high accuracy and robustness. Hollow meshes have been given smooth transitions between the solid and hollow regions, thus making them suitable for diverse applications in 3D modeling, CAD, and digital content creation.

4. RESULT

The model was trained on Objaverse 3D datasets with high-quality solid meshes in highly differentiated shapes and sizes, hence providing robust geometric presentation. The quality of the model was assessed using standard mesh quality metrics, such as Chamfer Distance and Intersection over Union (IoU), to evaluate its performance. The experimental results indicate that the model demonstrates superior performances in hollow mesh generation terms. Using the checkpoints of the CraftsMan system enables real-time hollow mesh synthesis, thus opening up completely new possibilities for interactive 3D modeling.

Solid Mesh

Hollow Mesh

5. DISCUSSION

Hollow mesh generation has thus proved a success for CRM, which can be used with SDFs and the dual-surface extraction algorithm. End-to-end optimization between the inner and outer structures allows the model to achieve geometric transformations that no other methods can successfully carry out. In addition, the incorporation of pre-trained models within the CraftsMan system makes the hollowing process speed much faster and more robust while being suitable for real-world applications in 3D modeling, gaming, and virtual reality.

The employment of CCMs ensures that CRM remains spatially coherent in the face of heterogeneity and complexity within objects. This structured methodology leads to a more granular understanding of spatial relations within 3D objects, thus resulting in good quality hollow outputs acquired.

CONCLUSION

This paper addresses the problem of the transformation of solid meshes into hollow meshes using the CraftsMan system. By using the Convolutional Reconstruction Model (CRM) that is embedded with state-of-the-art techniques like Flexicubes, Signed Distance Fields (SDFs), Dual Marching Cubes, and Canonical Coordinate Maps (CCMs), we present a flawless yet timely and strong method of having a hollow interlacement of a part. By using the checkpoints of the CraftsMan system, CRM establishes a new height of the hollow mesh synthesis, thus defining its feasibility across a range of industries that include 3D modeling, gaming, and virtual reality

What Can You Take Away from This White Paper