Kronos is a novel foundation model specifically designed for the language of financial markets, addressing the limitations of general-purpose Time Series Foundation Models (TSFMs) when applied to K-line data. The paper highlights that K-line sequences exhibit unique statistical properties, such as low signal-to-noise ratios, strong non-stationarities, and intricate high-order dependencies among OHLCVA (Open, High, Low, Close, Volume, Amount) attributes, which are often misaligned with generic TSFM inductive biases. Kronos aims to overcome these challenges by introducing a specialized, unified, and scalable pre-training framework.

The core methodology of Kronos operates in two distinct phases: (1) K-line Tokenization and (2) Autoregressive Pre-training.

1. K-line Tokenization:
This phase transforms continuous, D-dimensional K-line inputs $x = (x_1, \dots, x_T)$, where each $x_t \in \mathbb{R}^D$ (with $D=6$ for OHLCVA attributes), into a corresponding sequence of discrete tokens $b_{1:T} = (b_1, \dots, b_T)$. This is achieved using a Transformer-based autoencoder composed of an encoder $E_{enc}$, a quantizer $Q$, and a decoder $E_{dec}$.

• Quantization Mechanism: Kronos adapts Binary Spherical Quantization (BSQ), a variant of Look-up Free Quantization (LFQ), to quantize a continuous latent vector $\xi_t$ into a $k$-bit binary code $b_t \in \{-1, 1\}^k$.
• Hierarchical Token Structure: To manage the exponentially large vocabulary size ($2^k$) associated with large $k$, each $k$-bit token $b_t$ is factorized into two subtokens: a coarse subtoken $b_t^c$ and a fine subtoken $b_t^f$. These subtokens have equal bit lengths, $k_c = k_f = k/2$, such that $b_t = [b_t^c; b_t^f]$. This decomposition transforms a single prediction over $2^k$ entries into two sequential predictions over $2^{k/2}$ entries, significantly reducing computational and parameter complexity.
• Tokenization Objective: The tokenizer is trained with a composite objective that enforces a coarse-to-fine structure:

$L_{tokenizer} = L_{coarse} + L_{fine} + \lambda L_{quant}$

• $L_{coarse} = \mathbb{E}[\|x - E_{dec}(b^c)\|^2]$: This term encourages the coarse subtoken $b^c$ to capture the principal, low-fidelity structure of the input $x$ by minimizing the reconstruction error when only the coarse subtoken is used by the decoder.
• $L_{fine} = \mathbb{E}[\|x - E_{dec}(b)\|^2]$: This term ensures high-fidelity reconstruction using the complete token $b = [b^c; b^f]$. By optimizing $L_{fine}$ after $L_{coarse}$, the fine-grained subtoken $b^f$ is implicitly guided to encode the residual information necessary to refine the coarse approximation.
• $L_{quant}$: This is a quantization loss derived from BSQ, regularizing the learning process by penalizing the $L_2$ distance between continuous latent vectors $\xi$ and their binary codes $b$, promoting stable training and alignment with the learned codebook.

2. Autoregressive Pre-training:
In this phase, a decoder-only Transformer, denoted $E_{ar}$, is pre-trained on the tokenized sequences using a standard next-token prediction objective. $E_{ar}$ employs causal attention to ensure predictions at each time step depend exclusively on historical context.

• Hierarchical Autoregressive Modeling: Given the hierarchical token design, the conditional probability for each token $b_t$ is decomposed using the chain rule to explicitly capture the coarse-to-fine dependency:

$p(b_t|b_{<t}) = p(b_t^c|b_{<t}) \cdot p(b_t^f|b_{<t}, b_t^c)$.
This factorization allows the model to first predict the coarse-grained subtoken, which then serves as a condition for generating the fine-grained residual subtoken.

• Input Embedding: For each time step $i$, the coarse subtoken $b_i^c$ and fine subtoken $b_i^f$ are independently projected into vector representations using two distinct embedding layers, yielding $e_c(b_i^c)$ and $e_f(b_i^f)$. These embeddings are then concatenated and linearly projected to produce a fused input vector:

$v_i = W_{fuse}([e_c(b_i^c); e_f(b_i^f)])$, where $W_{fuse}$ is a learnable weight matrix.

• Prediction Process: The sequence of fused input vectors $\{v_1, \dots, v_{t-1}\}$ is processed by $E_{ar}$, yielding contextualized hidden states. The final hidden state $h_t$ (from processing $b_{<t}$) is used for prediction.
  • Coarse Subtoken Prediction: The hidden state $h_t$ is projected by a linear head $W_c$ to produce logits for the coarse subtoken distribution:

$p(b_t^c|b_{<t}) = \text{softmax}(W_c h_t)$.

• Fine Subtoken Prediction: To model the conditional dependency, the context is updated with the *predicted* coarse subtoken $\hat{b}_t^c$. This is achieved via a cross-attention mechanism where the embedding of $\hat{b}_t^c$ acts as the query, and $h_t$ provides the key and value:

$h_{update_t} = \text{CrossAttn}(q = e_c(\hat{b}_t^c), k = v = h_t)$.
Then, the updated hidden state $h_{update_t}$ is projected by a second head $W_f$ to predict the fine subtoken:
$p(b_t^f|b_{<t}, b_t^c) = \text{softmax}(W_f h_{update_t})$.
During training, $\hat{b}_t^c$ is sampled from the predicted distribution $p(b_t^c|b_{<t})$ (rather than using the ground truth) to mitigate exposure bias and align training with multi-step autoregressive inference.

• Training Objective: The overall autoregressive training objective $L_{ar}$ is the negative log-likelihood of the observed sequence, summed over both prediction steps:

$L_{ar} = -\mathbb{E}_{b \sim \mathcal{D}} \sum_{t=1}^{T} [\log p(b_t^c|b_{<t}) + \log p(b_t^f|b_{<t}, b_t^c)]$.

Kronos is pre-trained on a massive, diverse financial corpus of over 12 billion K-line records from over 45 global exchanges and 7 temporal granularities, enabling it to learn robust and generalizable market representations. This unified discretize-and-generate paradigm allows Kronos to construct a high-fidelity, hierarchical representation of market dynamics, serving as a robust foundation for various downstream quantitative finance tasks.