autopreso is an open-source, local web application designed to facilitate hands-free, speech-driven, AI-powered real-time whiteboarding, primarily leveraging Excalidraw. Its core purpose is to allow users to present and generate visual content on a digital whiteboard using spoken language, eliminating the need for manual interaction (clicking, typing) during a presentation. It is currently in an alpha stage, indicating active development.

The core methodology of autopreso revolves around a closed-loop system that processes audio input, transcribes it, feeds it to an AI agent, and uses the agent's outputs to dynamically modify an Excalidraw scene. The workflow, depicted as:
$audio (mic) -> text (STT) -> agent (tool calls) -> Excalidraw scene$
is detailed as follows:

1. Audio Input: A microphone (connected to the browser) captures audio at 24kHz.
2. Speech-to-Text (STT): The captured audio is streamed to a Speech-to-Text service. This can be a local Moonshine instance (a native sidecar for darwin-arm64 and darwin-x64 platforms) or OpenAI's Realtime transcription service via WebSocket. The STT engine converts the audio into text chunks.
3. Agent Processing: The transcribed text chunks are fed to a large language model (LLM) agent. Supported agent models include OpenAI's gpt-5.5 (or other OpenAI models), Codex (via subscription), or locally hosted Ollama models. This agent, leveraging the Vercel AI SDK for its tool-calling loop, interprets the spoken commands and generates "tool calls." These tool calls are instructions or commands formatted for the Excalidraw scene.
4. Excalidraw Scene Manipulation: The tool calls generated by the agent are executed against a live Excalidraw canvas. Excalidraw, serving as the whiteboard canvas, scene model, and rendering engine, receives these commands and updates the visual scene in real time, reflecting the spoken instructions. This creates a dynamic, evolving whiteboard display driven by speech.

The application operates in two distinct modes:

• Staging Mode: This client-side mode allows users to pre-sketch seed content (e.g., titles, agenda items) on the canvas before a presentation. It also serves as the interface for selecting microphone, transcription model, agent model, and providing optional Agent instructions.
• Live Mode: Upon initiating a presentation ("Start Preso"), the canvas control is handed over to the agent. The system biases OpenAI Realtime transcription towards text and labels already present in the staging content, and transcription streaming begins, driving the real-time drawing process.

autopreso runs as a local web app with an Express and WebSocket server, binding exclusively to 127.0.0.1, ensuring no external exposure. User settings, including selected models, API keys, STT choices, and agent instructions, are persisted across restarts in ~/.config/autopreso/settings.json. A "warmup loop" primes the agent with staging content and instructions before the presentation starts, improving initial response quality. The application also provides a "Session cost card" to estimate token costs for agent and OpenAI Realtime audio, resetting with each new presentation.

Installation methods include npm (globally or via npx), or building from source. Configuration is primarily managed via an in-app status panel, which overrides environment variables after the first run. Environment variables such as PORT, OPENAI_API_KEY, OPENAI_MODEL, CODEX_MODEL, and OLLAMA_MODEL can seed initial settings. Auto-detection prioritizes Codex, then Ollama, then OpenAI API keys for agent provisioning, with OpenAI Realtime transcription enabled if an OpenAI key is present, otherwise defaulting to Moonshine.