Open Models, Open Runtime, Open Harness - Building your own AI agent with LangChain and Nvidia
Video

Open Models, Open Runtime, Open Harness - Building your own AI agent with LangChain and Nvidia

2026.03.19
·YouTube·by 이호민
#AI Agent#LangChain#Nvidia#Open Source#Runtime

Key Points

  • 1This video outlines how to build open-source AI agents by integrating Nvidia's Neotron 3 Super Model, Nvidia's OpenShell runtime, and Langchain's Deep Agents harness.
  • 2OpenShell provides a secure, policy-governed execution environment with GPU acceleration, while Deep Agents orchestrates the agent's actions, manages context engineering, and connects to various file systems and execution environments.
  • 3This stack enables the creation of powerful agents with capabilities like secure code execution, file manipulation, and persistent memory outside the runtime sandbox, showcasing a complete open-source agent foundation.

An agent system, exemplified by tools like Claude Code, Manis, and OpenClaw, fundamentally comprises three core elements: a model, a runtime, and a harness, with an agent built on top. This paper details how to construct such an agent using an entirely open-source stack.

The proposed open-source stack consists of the NVIDIA Neotron 3 Super Model as the model, NVIDIA's OpenShell as the runtime, and LangChain's Deep Agents as the harness.

The Neotron 3 Super Model, released by NVIDIA, is highlighted for its superior performance in both accuracy and speed, surpassing OpenAI's GPTOS model. Its speed is particularly emphasized for enabling a highly responsive agent experience.

OpenShell, also from NVIDIA, serves as a secure runtime environment for agents. Its key features include robust security mechanisms, allowing for the definition of granular permission sets to control agent actions, and the ability to run locally, benefiting from GPU acceleration for tasks leveraging GPU-accelerated libraries. OpenShell sandboxes are policy-governed, enabling restricted environments (e.g., controlled network access).

Deep Agents is LangChain's open-source, model-agnostic agent harness designed to orchestrate models with various tools and environments. Its core functionalities include:

  • Tooling: Providing tools for connecting to runtimes, notably robust file system tools.
  • Skills and Subagents: Facilitating deeper, specialized agent capabilities.
  • Context Engineering: Managing common context window challenges like summarization, abstracting these complexities from the developer.
  • Backends: Deep Agents utilizes an abstraction called "backends" to allow agents to connect to diverse file systems and execution environments.

The methodology for building and demonstrating this agent involves several steps:

  1. OpenShell Setup: Initiating the OpenShell gateway and creating a persistent sandbox (e.g., deep_agent_sandbox) for consistent execution environments.
  2. Environment Configuration: Setting up environment variables in an .env file, including the NVIDIA API key for model access, the OpenShell sandbox name, and optionally, Langsmith API keys for agent observability and evaluation.
  3. Agent Definition (main.py): The agent's core is defined by:
    • Model Initialization: Specifying the NVIDIA Neotron 3 model and loading the API key.
    • System Prompt: A fixed, foundational instruction set defining the agent's capabilities, workflow, and sandbox interaction guidelines.
    • Agent Memory: A separate file (agent.mmd) containing base-level instructions that the agent can dynamically update over time, unlike the fixed system prompt.
    • Backend Creation: Utilizing a create_backend function that leverages Deep Agents' CompositeBackend. This crucial component layers different backend types:
      • An OpenShellBackend connected to the active OpenShell sandbox session for command execution.
      • A FileSystemBackend to mirror local memory and skills files into the agent's environment, enabling persistent memory across sessions or sandboxes.

The demonstration of the agent's capabilities via the langgraph dev CLI utility and Langsmith Studio's interface illustrates:

  • Basic Interaction: Handling simple chat prompts.
  • Sandbox Execution: Running commands within the OpenShell sandbox (e.g., uname -a, python3 --version).
  • File Operations: Demonstrating the agent's ability to write a Python script to a file, execute it within the sandbox, and process its output.
  • Policy Enforcement: Testing OpenShell's security features by attempting a network request to evil.com, which is blocked by the sandbox's policy, showcasing its restricted environment capabilities.
  • Memory Update: Showing how the agent can modify its agent.mmd memory file (e.g., updating its language preference to Spanish), demonstrating the persistence of memory outside the execution sandbox.

The paper concludes by emphasizing the ongoing investment in Deep Agents as an open-source harness, with future plans including asynchronous subagents and diverse execution environments. It also highlights the potential of OpenShell, particularly when deployed on GPU-accelerated boxes, to enable agents to perform complex, GPU-dependent data processing tasks, alongside continued excitement for the progression of open-source models.