GitHub - ideogram-oss/ideogram4: Ideogram 4: Open image model at the forefront of design
Key Points
- 1Ideogram 4 is Ideogram's first open-weight text-to-image model, a 9.3B parameter Diffusion Transformer trained from scratch, featuring a vision-language model as its text encoder.
- 2It introduces a structured JSON prompting interface for unprecedented control over composition, style, and layout, enabling best-in-class multilingual text rendering, explicit bounding-box placement, and color palette conditioning.
- 3Benchmarked as a leading open-weight model, Ideogram 4 delivers state-of-the-art performance in design-oriented generation, superior text rendering, and layout control across various third-party and internal evaluations.
Ideogram 4 is an open-weight, state-of-the-art text-to-image foundation model trained from scratch, not a fine-tune of any existing model. With 9.3 billion parameters, it introduces a novel structured JSON prompting interface, enabling best-in-class multilingual text rendering, deep language understanding, explicit bounding-box layout and color-palette controls, and native 2k resolution images.
The core methodology of Ideogram 4 is built upon a flow-matching text-to-image approach utilizing a fully single-stream Diffusion Transformer (DiT) architecture. Unlike traditional multi-branch models, Ideogram 4 concatenates text and image tokens into one unified sequence. This combined sequence is then processed through a single, deep 34-layer transformer. This single-stream design facilitates deep cross-modal interaction at every layer, allowing the model to intricately fuse textual and visual information throughout the generation process.
For its text encoding, Ideogram 4 departs from standard text-only encoders like CLIP or T5. Instead, it employs Qwen3-VL-8B-Instruct, a full vision-language model (VLM). This VLM acts as the text encoder, providing significantly richer understanding of visual concepts and contexts embedded within the prompt. Hidden states are extracted from 13 intermediate layers of Qwen3-VL-8B-Instruct and then concatenated. This concatenation provides the DiT with multi-scale semantic features, ranging from surface-level token information to deep compositional understanding, enhancing the model's ability to interpret complex prompts.
The model incorporates dual-branch classifier-free guidance, allowing for independent refinement of conditional (positive) and unconditional (negative) branches. This enables separate, fine-grained control over prompt adherence and overall image quality during inference.
A key architectural feature is its flexible resolution support. Ideogram 4 natively handles any resolution from 256 to 2048 pixels (in multiples of 16), with aspect ratios up to 6:1. A single model handles this entire range, from square thumbnails to ultrawide banners, with the noise schedule automatically adjusting per resolution to maintain quality.
Ideogram 4's primary innovation in user control comes from its structured JSON caption training. The model is exclusively trained on JSON-formatted captions that are extremely descriptive, exhaustively detailing every element within an image. This training paradigm maximizes efficiency by grounding numerous text-to-image relationships from each training pair. During inference, this JSON-only training allows for unprecedented control through:
- Color palette conditioning: Users can specify hex colors within the JSON prompt's style description to steer the image's dominant color scheme.
- Spatial layout control: Explicit bounding-box coordinates (
bbox) can be defined in the prompt to precisely place subjects, text elements, and background regions. - Compositional control: The
compositional_deconstructionfeature, combined with bounding boxes and per-element descriptions, facilitates precise spatial arrangement of multiple distinct elements.
While plain-text prompts are supported, the model performs optimally with structured JSON inputs due to its training regimen. To bridge this gap, a "magic prompt" LLM automatically expands plain-text prompts into full structured JSON captions server-side, enabling JSON-quality results from casual inputs.
Performance evaluations show Ideogram 4 leading open-weight models in design-oriented benchmarks (Design Arena, ContraLabs) and general-purpose text-to-image leaderboards (LMArena). It also significantly closes the gap to leading closed-source models across standard open-source benchmarks for layout control (7Bench), spatial reasoning (SpatialGenEval), text rendering (X-Omni OCR), and prompt alignment (Prism), delivering best-in-class text rendering among open-weight models.