The "HTML in Canvas" API is introduced as a novel bridge between the Web Platform's two primary rendering paradigms: the Document Object Model (DOM) and the Canvas (including WebGL and WebGPU). The core problem addressed is that while the DOM excels at rich, interactive, accessible UI with inherent browser features (text selection, copy, right-click, accessibility, translate, find in page, reader mode, extensions, dark mode, browser zoom, autofill), Canvas offers low-level, high-performance 2D and 3D graphics but requires developers to manually implement complex UI logic like text layout and forfeits most browser-level features. The HTML in Canvas API aims to deliver the best of both worlds by allowing DOM content to be drawn directly into a Canvas or WebGL/WebGPU texture while preserving full interactivity, accessibility, and integration with native browser functionalities.

The core methodology of the HTML in Canvas API involves a three-phase process: setup, rendering, and transform update.

1. Setup: The foundation is laid by declaring the intent to embed HTML within the canvas. This is achieved by setting the layout-subtree CSS property on the target $<canvas>$ element. This property signals to the browser that content within this canvas should be considered for hit testing, accessibility, and other browser-level features. HTML content (e.g., $<form>$, $<input>$) is then written directly as a child of the $<canvas>$ tag, either declaratively in the HTML source or programmatically via JavaScript. An important step to avoid blurriness is to correctly set the width and height of the canvas to the right scale factor, often managed with a ResizeObserver.

2. Rendering: Once the HTML content is placed within the canvas, it must be explicitly rendered into the canvas's drawing buffer or a GPU texture.
   • 2D Context: For a standard 2D rendering context, the method context.drawElementImage(domElement, x, y) is used. This function takes a reference to the DOM element to be drawn and its target x and y coordinates within the canvas. The speaker recommends executing this call within an onpaint event handler, which triggers whenever the element requires redrawing (e.g., due to user interaction or browser features like text highlighting). This ensures the UI remains continuously updated.
   • WebGL Context: For WebGL, the API provides gl.textElementImage2D(). This method mirrors the existing gl.texImage2D but, instead of an image source, accepts a reference to the DOM element to be rendered into a texture.
   • WebGPU Context: Similarly, for WebGPU, the method device.copyElementImageToTexture() is available. This function is analogous to WebGPU's copyExternalImageToTexture and allows rendering a DOM element directly into a WebGPU texture.

3. Transform Update: After rendering the DOM element into the canvas or a texture, it is critical to inform the browser of the element's rendered position and orientation within the overall document flow. This is achieved by applying a CSS transform to the original DOM element.
   • For 2D contexts, the drawElementImage call returns a transform, which is then directly applied to the CSS transform property of the DOM object. This step is crucial for enabling interactivity and browser functionality, as it allows the browser's hit-testing and accessibility systems to correctly map user interactions and feature requests to the rendered content. The speaker emphasizes the necessity of updating this transform whenever the element is repainted. If the element is no longer painted, it must be removed from the canvas element's children to prevent it from being presented to accessibility systems.
   • For 3D contexts (WebGL/WebGPU), the process is slightly more complex. A viewport transformation is applied to map the 3D coordinate space of the rendered element to the 2D CSS coordinate space. The canvas.getElementTransform() method is then used to obtain a transform that can be applied to the DOM element's CSS transform property, similar to the 2D case.

The API is designed for seamless integration with existing frameworks. Three.js has experimental support via THREE.HTMLTexture, which takes a DOM element and maps it as a material texture. PlayCanvas also supports the API by allowing an HTML element to be set as the source of a texture, which can then be applied to materials. These frameworks abstract the complex 3D transform calculations.

Key benefits highlighted include:

• Full Interactivity: Input fields, buttons, and other UI elements remain fully functional.
• Accessibility: Content rendered within the canvas is exposed to the accessibility tree, allowing screen readers and other assistive technologies to interact with it.
• Browser Feature Integration: All standard browser features (translate, find in page, dark mode, zoom, autofill, etc.) work as expected on the rendered content.
• Reduced Development Overhead: Developers can leverage the browser's highly optimized text and UI layout engines instead of implementing custom solutions, potentially reducing bundle size.
• SEO and AI Indexing: Because the content is still part of the DOM tree, it can be easily indexed by search engines and processed by AI agents, enabling text extraction and other functionalities.

The API is currently available in Chrome as an Origin Trial (as of May 2026, implying a future date in the video or a typo in the transcript, likely referring to May 2023 when the video was published), requiring sign-up for production use. It is actively being discussed as an open standard with other browser vendors. The presentation demonstrates various practical applications, including dynamic UI overlays on 3D models, interactive 3D books with live font changes and translation, animated SVGs within 3D scenes, and refractive UI elements.