TRELLIS.2 is a state-of-the-art, large 3D generative model with 4 billion parameters, primarily designed for high-fidelity image-to-3D generation. It introduces a novel "field-free" sparse voxel representation called O-Voxel, enabling the reconstruction and generation of complex 3D assets with arbitrary topologies, sharp features, and full PBR (Physically Based Rendering) materials.

The core methodology of TRELLIS.2 is built upon a multi-stage generative process that leverages sparse, structured latent representations and vanilla Diffusion Transformers (DiTs).

1. O-Voxel Representation:
At its foundation, TRELLIS.2 utilizes O-Voxel, a sparse voxel structure that differs from traditional iso-surface fields. This representation allows for:

• Robust handling of arbitrary topologies, including open surfaces (e.g., clothing), non-manifold geometry, and internal enclosed structures, without requiring lossy conversion processes.
• Direct modeling of rich texture attributes such as Base Color, Roughness, Metallic, and Opacity, facilitating photorealistic rendering and transparency.

O-Voxel provides highly efficient bidirectional conversion: textured meshes can be converted to O-Voxels in less than 10 seconds (CPU), and O-Voxels to textured meshes in under 100 milliseconds (CUDA).

2. Sparse 3D VAE for Latent Encoding:
3D assets are encoded into a compact, native, and structured latent space using a Sparse 3D Variational Autoencoder (VAE). This VAE performs a 16x spatial downsampling, mapping high-resolution O-Voxel data into a significantly smaller, more manageable latent representation. This compressed latent space is crucial for efficient generation and manipulation.

3. Multi-stage Generative Flow Models with DiTs:
The generation process is structured into multiple diffusion-based flow models, primarily implemented using vanilla DiTs, which operate on these structured latents:

• SC-VAE Training (Latent Space Learning):
  • Shape SC-VAE: Learns the latent distribution for 3D shapes from raw 3D assets (converted to O-Voxel, then dual-grid representations).
  • Texture SC-VAE: Learns the latent distribution for PBR textures, operating on PBR dumps and voxelized PBR data.

• Flow Model Training (Conditional Generation):

These models progressively generate more detailed latent representations based on input conditions:

• Sparse Structure Flow Model (Image to Sparse Structure Latent): This initial stage takes a conditional rendering image and generates a sparse structure latent representation. This directly connects the input image to the fundamental sparse O-Voxel characteristics.
  • Configuration: ss_flow_img_dit_1_3B_64_bf16.json
  • Data: ss_latent, render_cond
• Shape Flow Model (Image to Shape Latent): Building upon the sparse structure, this model generates the compact shape latent from the input image.
  • Configuration: slat_flow_img2shape_dit_1_3B_512_bf16.json
  • Data: shape_latent, render_cond
• Texture Flow Model (Image + Shape to PBR Latent): This final generative stage combines the input image and the generated shape latent to predict the PBR texture latent.
  • Configuration: slat_flow_imgshape2tex_dit_1_3B_512_bf16.json
  • Data: shape_latent, pbr_latent, render_cond

These flow models operate within the learned latent spaces, progressively refining the 3D asset's shape and material properties. Higher-resolution fine-tuning stages are also supported (e.g., for 1024³ resolution) by adjusting finetune checkpoints and dataset paths.

Performance and Features:
TRELLIS.2 achieves exceptional fidelity and efficiency, generating high-resolution, fully textured assets.

• Resolution & Time (H100 GPU):
  • 512³: ~3s (2s Shape + 1s Material)
  • 1024³: ~17s (10s Shape + 7s Material)
  • 1536³: ~60s (35s Shape + 25s Material)
• Arbitrary Topology Handling: Overcomes limitations of iso-surface fields by directly supporting open surfaces, non-manifold geometry, and internal structures through O-Voxel.
• Rich Texture Modeling: Beyond basic colors, it models Base Color, Roughness, Metallic, and Opacity.
• Streamlined Data Processing: Fast and optimization-free conversions between textured meshes and O-Voxel.

The model is built upon specialized high-performance packages, including O-Voxel for core representation logic, FlexGEMM for efficient sparse convolution, and CuMesh for CUDA-accelerated mesh utilities.