DiffusionGemma is an experimental, open-source multimodal language model based on the Gemma 4 architecture, specifically a 26-billion parameter Mixture-of-Experts (MoE) variant with 4 billion active parameters. It pioneers the exploration of text diffusion, diverging from traditional causal token generation methods.

The core methodology of DiffusionGemma revolves around discrete text diffusion coupled with block autoregressive multi-canvas sampling. Instead of generating tokens sequentially, the model iteratively denoises blocks of tokens, referred to as "canvases," in parallel. This parallel generation of token blocks (e.g., 256 tokens at once) significantly enhances decoding speed. The architecture is an encoder-decoder system: a self-regressive encoder processes and caches prompt context, while the denoising process applies bidirectional attention to the generation canvas.

DiffusionGemma features multimodal processing, capable of handling diverse inputs including text, images (supporting various aspect ratios and resolutions), and video, to generate text outputs. It does not support audio input. The model leverages a sparse MoE design, offering deep reasoning capabilities with minimal overhead. Its efficiency is further highlighted by quantization, allowing it to fit within 18GB VRAM, making it suitable for local execution on consumer GPUs. An integrated thought mode provides configurable inference channels, enabling the model to perform step-by-step reasoning before producing a final answer.

In contrast to conventional language models optimized for large-scale cloud deployments with high batching efficiency, DiffusionGemma is designed to maximize local hardware performance for single-user scenarios. It addresses the underutilization of hardware in local execution by generating entire blocks of tokens simultaneously rather than one token at a time. While this approach significantly boosts performance for concurrent, low-batch-size local usage, its throughput benefits diminish in high Query Per Second (QPS) cloud environments due to the nature of parallel decoding.

For optimal latency and quality, recommended serving configurations include:

• A maximum of 48 denoiser steps, although adaptive stopping typically results in 12-16 steps depending on task complexity.
• A temperature schedule that linearly decreases from 0.8 to 0.4 across denoising steps, promoting initial exploration (0.8) and subsequent token fixing (0.4).
• Adaptive early stopping based on an entropy threshold of 0.005, where execution halts if the average model entropy across the canvas falls below the threshold and two consecutive denoiser predictions are identical. This dynamically adjusts speed based on task complexity.
• A token selection entropy boundary of 0.1, ensuring that at each step, the sampler selects only tokens with the lowest entropy (i.e., tokens the model is most certain about), allowing other less certain tokens to be further denoised in subsequent steps.