thinkingmachines/Inkling-Small · Hugging Face
Key Points
- 1Inkling-Small is a 276B parameter multimodal Mixture-of-Experts transformer designed to process text, image, and audio inputs for diverse applications like coding, agentic tasks, and conversational AI.
- 2The model demonstrates high performance across key benchmarks, achieving competitive results on evaluations like SWE-bench Verified (80.2%) and AIME 2026 (95.5%) while maintaining efficient inference through its 12B active parameter architecture.
- 3Released with open weights under an Apache 2.0 license, the model includes built-in safety mitigations and is compatible with standard moderation tools, though developers are advised to implement additional application-layer safeguards for high-stakes use cases.
Inkling-Small is a general-purpose, multimodal, autoregressive transformer model with a total parameter count of 276 billion, featuring 12 billion active parameters. The architecture is built upon a 42-layer decoder-only transformer that utilizes a sparse Mixture-of-Experts (MoE) feed-forward backbone. In this configuration, each input token is processed by routing it to 6 out of 256 available experts, supplemented by 2 shared experts that remain active for every token. The attention mechanism incorporates a hybrid structure, alternating between local and global attention layers to optimize performance.
The model is natively multimodal, designed to ingest text, images, and audio, and map them into a shared hidden space for joint processing. Images are handled via a hierarchical patch encoder, while audio inputs (WAV format at 16kHz) are processed using discrete token encoding. Output is generated as UTF-8 encoded text. The model supports BF16 and NVFP4 numeric formats and is optimized for deployment via frameworks such as SGLang, vLLM, TokenSpeed, and Unsloth.
Training involved a large-scale curation of diverse datasets—including text, images, audio, and video—sourced from the public internet and third-party providers. The process utilized rigorous deduplication and filtering to improve data quality and maintain safety standards.
Performance benchmarks highlight the model's high proficiency in agentic and reasoning tasks. Key results include:
- Coding and Agentic Tasks: Achieved 80.2% on SWE-bench Verified and 55.9% on SWE-bench Pro.
- Reasoning: Recorded 89.5% on GPQA Diamond and 95.5% on the AIME 2026 mathematics benchmark.
- Multimodal Capabilities: Scored 74.0% on MMMU Pro (Standard 10 Options) and demonstrated strong audio-processing performance, achieving 90.1% on VoiceBench.
Safety evaluations prioritized consistency across modalities, assessing sycophancy, harmful manipulation, and psychological-harm patterns. While the model showed latent capability in cyber and CBRN domains, it remained materially below frontier levels. The researchers suggest that residual risks, such as a tendency to comply with role-play-framed harmful requests, should be addressed via defense-in-depth strategies, including layering the model with external moderation tools like Llama Guard. The paper acknowledges inherent limitations common to foundation models, including hallucinations, potential biases in training data, and a knowledge cutoff, recommending that users implement human oversight for high-stakes, safety-critical, or professional applications.