Seed3D 2.0 is a next-generation 3D generative model that significantly advances the creation of high-quality, large-scale 3D content from a single image, targeting production-grade requirements for geometric precision and material realism. Building upon Seed3D 1.0, this iteration focuses on an upgraded architecture to achieve state-of-the-art (SOTA) results in geometry and texture/material generation, while also expanding downstream usability.

For Geometry Generation, Seed3D 2.0 introduces a Coarse-to-Fine, two-stage generation strategy to decouple and optimize "overall structure" and "fine details," addressing the "softening" effect and imprecision encountered in previous methods.

1. Stage 1: Coarse Geometric Structure Generation employs a Diffusion Transformer (DiT) with a larger parameter scale to generate coarse-grained geometric structures from the input image, establishing the fundamental topological relationships and spatial layout.
2. Stage 2: High-Precision Detail Generation leverages the output of Stage 1 as geometric anchors to recover intricate details like sharp edges and refined surfaces. This stage integrates two key priors:
   • Local-aware prior: The coarse results from Stage 1 are converted into latent variables, providing a reliable initialization to avoid instability from "generating from scratch."
   • Voxelized positional encoding: Points sampled on the geometric surfaces from Stage 1 are voxelized and serve as positional encodings, supplying critical spatial constraints to the model.

Additionally, the underlying Variational Autoencoder (VAE) in Seed3D 2.0 has been upgraded to achieve higher reconstruction fidelity with fewer tokens, enhancing local detail expression and inference efficiency through dynamic attention allocation. Qualitative and human evaluations confirm Seed3D 2.0's superior performance in rendering complex geometries, thin-walled structures, and maintaining input image fidelity.

For Texture Generation, Seed3D 2.0 shifts from a cascaded RGB generation and PBR decomposition model (which accumulated errors in Seed3D 1.0) to a unified PBR generative model. This model retains the MMDiT dual-stream architecture, utilizing modality-specific projection layers to jointly process the full set of PBR texture maps within shared DiT layers, ensuring physically consistent visual effects under varying lighting. Two key innovations contribute to precise, high-resolution material generation:

3. Mixture of Experts (MoE) architecture: To generate richer texture details and accurate metal-roughness boundaries without prohibitive computational cost, an MoE architecture is adopted. This expands model parameters and resolution while controlling inference computation through sparse expert routing.
4. Leveraging VLM priors: To bolster the stability and accuracy of material decomposition, especially under unseen lighting conditions, a Vision-Language Model (VLM) generates descriptions of material types and physical properties from the input image. These descriptions are injected into the DiT as additional control signals, making the material decomposition more robust and reasonable, addressing issues like color shifts or incorrect metallic classifications. Qualitative assessments and human evaluations consistently show Seed3D 2.0's superiority in photorealism, material quality, and visual detail.

Beyond core generation, Seed3D 2.0 expands Downstream Applications:

• Part-Level Generation: It adopts an "understand-then-generate" paradigm for decomposing generated 3D content into functional components. The Seed3D-PartSeg 3D comprehension module learns to partition 3D data by performing surface segmentation on full 3D meshes. Building on this, Seed3D-PartDiT uses global 3D shape, segmented point clouds, and images as inputs to complete individual part shapes, assembling them into full part-level meshes.
• Articulated Modeling: Building on part segmentation, Seed3D 2.0 incorporates articulated modeling. VLMs decompose parts into kinematic components and identify joint types (e.g., revolute), estimating joint axes via geometric priors. An image-to-video model generates motion references to optimize the range of motion. The model outputs 3D content with complete joint information in standard formats like URDF, compatible with physics simulation engines (e.g., Isaac Sim).
• Scene Composition: Extending from single-object generation, Seed3D 2.0 generates complete scenes. For text inputs, a fine-tuned Large Language Model (LLM) handles spatial reasoning and layout generation. For multi-view image or video inputs, visual signals like depth estimation, instance segmentation, and occlusion inpainting infer scene layouts. Once layouts are determined, individual 3D content is generated and assembled according to spatial relationships. The integration of part-level generation and articulated modeling allows generated scene objects to support physical interactions.

While Seed3D 2.0 marks significant advancements, the paper acknowledges ongoing challenges in 3D generation, including further precision improvements, generalization capabilities, addressing occlusion and mapping errors in texture generation, and inference efficiency for large-scale deployment.