DFlash 2 is an advanced inference optimization framework designed to accelerate large language model (LLM) decoding for agentic workloads. Building upon the original DFlash architecture, which enabled one-pass, parallel drafting of token blocks, DFlash 2 introduces a path-selection mechanism and dynamic local convolutions to address the issues of token incoherence and "suffix decay" (a degradation in prediction accuracy toward the end of a drafted block).

Core Methodology

1. Parallel Path Selection
The primary limitation of independent parallel drafting is that while individual tokens are likely correct, the sequence of tokens often lacks local coherence, causing the target model to reject the block. DFlash 2 addresses this by treating the top 16 candidate tokens at each position as a state space and selecting a single coherent path. It defines a score for each adjacent token pair $(a, b)$ at position $t$ using:
$$S_t(a,b) = U_t(b) + \langle A(a) \odot H(h_t), B(b) \rangle$$
Here, $U_t(b)$ represents the drafter’s native logit, while the second term uses low-rank bilinear attention—modulated by a context gate $H(h_t)$—to score the transition from predecessor $a$ to candidate $b$. This allows the system to trace the most probable coherent path through the candidates in one parallel operation, significantly increasing the likelihood of full block acceptance.

2. Dynamic Short Convolutions for Suffix Decay
Suffix decay occurs because deeper positions in a draft block have fewer dependencies on context and struggle with long-range coherence. Rather than increasing model depth (which adds prohibitive latency), DFlash 2 employs two-tap dynamic depthwise convolutions at each layer. This mechanism mixes a token's representation with its predecessor's:
$$\operatorname{Conv}_{k}(x)_t = k_{t,0} \odot x_t + k_{t,1} \odot x_{t-1}$$
The first position reads the last verified token from the target model, and subsequent positions read the preceding drafted token. By forcing the drafter to explicitly model local dependencies through these lightweight convolutions (adding only 3% to parameter count and 0.7% to cycle latency), the framework effectively shifts the workload of "within-block" dependencies away from standard attention heads, which can then refocus on reading the global context.

Performance and Implementation

DFlash 2 maintains the "one-pass" design, ensuring that the target model verifies the entire block in a single forward pass. Experimental results on models like Qwen3.8-27B and Muse Glimmer show that DFlash 2 increases acceptance length by 16–25% over the original DFlash and significantly outperforms other speculative decoding methods like DSpark and MTP.

The framework is highly efficient, providing throughput gains of 2.7–4.6× compared to autoregressive decoding, depending on the model and hardware. It is currently integrated into major inference engines including SGLang, vLLM, llama.cpp, Ollama, and oMLX, making it a drop-in component for modern high-performance inference stacks. The combination of the path selector and the convolutional architecture allows DFlash 2 to push speculative decoding performance toward theoretical oracle limits while keeping cycle latency negligible.