raea (Render Anything Engine) is a high-performance, open-source engine designed to bridge the gap between AI-driven content generation and GPU-accelerated rendering. It allows developers and AI agents to define videos, motion graphics, 3D scenes, and interactive games using familiar web technologies—specifically React and TypeScript—and render them directly on local hardware.

Core Methodology and Architecture

The engine is built in Rust using the Bevy game engine (v0.20) as its foundation. Its architecture is strictly layered to decouple high-level abstractions from low-level GPU operations:

1. Composition Engine: At the runtime level, raea leverages a JavaScript engine (JavaScriptCore on macOS; QuickJS on Linux/Windows) to execute React code. A composition is defined as a component that resolves state over a sequence of frames, where $f$ represents the current frame index. Animation logic often relies on interpolation functions to map input domains to output ranges:

$$\text{output} = \text{interpolate}(f, [start, end], [min, max])$$

2. Kernel and GPU Pipeline: The core/ module defines a kernel language that translates high-level rendering instructions into WGSL (WebGPU Shading Language). This allows for efficient execution across different GPU architectures.
3. Systems Layer: The engine modularizes complex tasks into specific systems, including:
   • Layout & Compositing: Translates React-style styling (including Tailwind) into GPU-accelerated draw calls.
   • Media Processing: Handles video encoding/decoding via hardware-accelerated APIs like VideoToolbox, alongside software-based AV1 and muxing utilities.
   • Machine Learning Integration: Uses ONNX Runtime for local AI tasks, such as foley generation and instrument synthesis, enabling the engine to perform intelligent tasks like automated video cutting based on audio content.
   • 3D Rendering: Integrates Bevy-native features such as glTF model loading, Gaussian splatting, and programmable lighting/camera systems.

Agent-Centric Design

raea is engineered to be an "agent-first" tool. Unlike traditional renderers, it treats the rendered output as queryable data. Agents can:

• Render and Iterate: Call raea render to generate frames, inspect them, and utilize programmatic feedback loops to refine the composition.
• Quality Assurance: Use raea check to validate assets against a technical "floor" (e.g., checking for clipped text, low contrast, or unsafe broadcast areas).
• Skill Integration: The engine installs directly into agent environments (such as Claude Code) via a plugin system, allowing agents to manipulate the sdk/raea TypeScript library to programmatically generate complex visual outputs.

Technical Implementation

• Memory and Performance: By utilizing Rust and Bevy, raea achieves near-native performance. On Apple Silicon, for instance, a 90-frame composition can be rendered in approximately one second.
• Offline First: The engine is designed to minimize external dependencies. It enforces a strict "no telemetry" policy, and the $RAEA_OFFLINE=1$ flag ensures no network requests are made, keeping sensitive rendering tasks local to the user's GPU.
• Build Environment: The system relies on a complex toolchain including bun for the TypeScript SDK management, CMake for build orchestration, and a collection of low-level libraries for system integration (e.g., libvulkan-dev, libfontconfig1-dev).

The engine is distributed under the Functional Source License (FSL-1.1-ALv2), which allows free usage and modification for non-competitive purposes, with a transition to Apache-2.0 after two years.