GitHub - Leonxlnx/earshot: Ears for coding agents: let AI hear, review and make sound. CPU-only, open source.
Service

GitHub - Leonxlnx/earshot: Ears for coding agents: let AI hear, review and make sound. CPU-only, open source.

Leonxlnx
2026.11.16
·GitHub·by Mineru
#Agent#AI#LLM#ML#Transformer

Key Points

  • 1Earshot is an open-source, CPU-only toolkit that enables AI coding agents to hear, critique, and synthesize sound effects for digital media.
  • 2The platform provides a suite of CLI and API tools for audio analysis, ranging from descriptive labeling and search-based matching to a programmatic synthesis engine called "forge" that allows agents to create and refine sounds using code.
  • 3By offering deterministic rendering and professional-grade audit metrics, Earshot helps agents automate the design of realistic, perfectly synced sound kits for films, products, and games without relying on network-dependent AI models.

Earshot is an open-source, CPU-only toolkit designed to provide AI coding agents with auditory perception, enabling them to evaluate, critique, and synthesize sound effects for digital media like product demos, films, and games. It functions as a CLI, Python API, and MCP server, bridging the gap between generative code and high-quality audio output.

Core Functionality and Methodology

Earshot translates audio files into actionable evidence through several primary mechanisms:
  • Acoustic Analysis: The tool performs objective measurements (LUFS, true peak, spectral balance, and envelope) and uses pre-trained models—specifically CLAP (laion/larger_clap_generallaion/larger\_clap\_general) for text-audio embeddings and an AudioSet-finetuned AST (MIT/ast−finetuned−audioset−10−10−0.4593MIT/ast-finetuned-audioset-10-10-0.4593)—to generate calibrated tags, realness scores, and listening transcripts.
  • Evaluation and Critique: It provides "critique" pipelines for launch films by comparing the audio profile of generated content against professional benchmarks. It identifies timing issues (e.g., sync between sound and video cuts) and spectral deficiencies.
  • Synthesis via Analysis-by-Synthesis: The earshot fit module decomposes a target recording into a series of physical synthesis parameters. It fits models to the audio using techniques such as modal resonators, noise gestures, and tracked partials, ultimately outputting a Python-based "recipe" that allows for deterministic re-rendering and morphing.
  • The Forge (Sound Design): earshot forge enables agents to generate sounds from scratch using code-defined recipes. It employs an evolutionary search algorithm that optimizes parameters against a multi-faceted "critic." This critic scores candidates based on description match (cosine_similaritycosine\_similarity), tonal balance (specifically the 1/3-octave energy distribution), and model-free penalties that monitor for artifacts like comb-filtering, clipping, or truncated tails.

Synthesis and Sound Kits

The repository includes a bank of 252 synthesized sound effects categorized into 34 families (e.g., UI pops, knocks, mallet notes, camera shutters). The earshot kit system facilitates the creation of comprehensive sound designs for films:
  • Deterministic Rendering: Sound effects are rendered at 24-bit depth with measured onset, peak, and end points.
  • The "Launch" Preset: This preset coordinates the mix between sound effects and a music bed. It calculates relative gain levels (gaindBgain_{dB}) based on the music's median level (beddBbed_{dB}):
gaindB=take_level−beddBgain_{dB} = \text{take\_level} - bed_{dB}
  • Sync Logic: The system automates the spatial and temporal placement of effects against visual cues, enforcing constraints such as minimum gaps between rapid UI sounds (min_gapmsmin\_gap_{ms}) to prevent masking.

Technical Infrastructure

Earshot prioritizes local processing, eschewing cloud-based inference for offline, CPU-bound execution. It handles model loading via a pinned, cache-first architecture. The system supports custom environment configurations, such as EARSHOT_THREADSEARSHOT\_THREADS for managing CPU resource allocation and EARSHOT_PRECISIONEARSHOT\_PRECISION for selecting between bf16bf16 (on supported hardware) and fp32fp32. The toolkit provides robust validation through deterministic check-summing and audit-based verification, ensuring that renders meet specific acoustic quality standards before being integrated into a project.