The Intelligent OS: Making AI agents more helpful for Android apps
Blog

The Intelligent OS: Making AI agents more helpful for Android apps

2026.03.11
·Web·by 이호민
#AI Agents#Android Development#AppFunctions#Gemini#UI Automation

Key Points

  • 1Android is evolving its platform to enable AI agents like Google Gemini to perform tasks within apps, shifting user interaction from manual steps to AI-driven assistance.
  • 2The new AppFunctions Jetpack library allows developers to expose app data and functionality directly to AI agents via self-describing functions, enabling deep integrations for specific use cases.
  • 3Additionally, an intelligent UI automation framework is being developed to let AI agents execute generic multi-step tasks across apps without developer code, with built-in user transparency and control.

This paper details Google's strategy to evolve the Android ecosystem for an agentic future, where AI agents like Google Gemini perform complex tasks for users, shifting the focus from app opens to successful task completion. Two primary developer capabilities are introduced to bridge apps with AI agents: AppFunctions and intelligent UI automation, both prioritizing privacy, security, and user control.

AppFunctions
AppFunctions provide a structured framework for applications to expose their data and functionality directly to AI agents and assistants. Developers leverage the AppFunctions Jetpack library and platform APIs to create "self-describing functions." These functions, declared in a manner analogous to backend capabilities defined via MCP (Managed Compute Platform) cloud servers (or specifically WebMCP for web services), are designed for on-device discovery and execution. When a user issues a natural language query, an AI agent, such as Gemini, intelligently identifies and triggers the relevant AppFunction locally on the device, rather than on a remote server. This allows the AI agent to directly invoke app-specific functionalities and retrieve data, integrating the results seamlessly into the agent's interface without requiring the user to navigate the source application. A prime example is the Samsung Gallery integration with Gemini on the Galaxy S26 series, where a user query like "Show me pictures of my cat from Samsung Gallery" triggers an AppFunction to fetch and display photos directly within the Gemini app. AppFunctions also enable Gemini to automate tasks across categories like Calendar, Notes, and Tasks from various manufacturers. This capability is currently available on the Galaxy S26 series and is slated for expansion to Samsung devices running OneUI 8.5 and higher.

Intelligent UI Automation
Complementing AppFunctions, an intelligent UI automation framework is being developed to enable AI agents to execute generic, multi-step tasks on users' installed applications, particularly for interactions lacking a dedicated AppFunctions integration. This framework is designed to perform the "heavy lifting" at the platform level, allowing developers to extend their app's reach to agentic experiences with zero code changes. It provides intelligent task execution by leveraging the context already available from user apps. The system ensures user transparency and control; users can monitor task progress via notifications or a "live view" and can switch to manual control at any point. Furthermore, Gemini is designed to alert users before completing sensitive actions like purchases. This feature is currently in an early preview beta, accessible via a long press of the power button on the Galaxy S26 series and select Pixel 10 devices, and supports a curated selection of apps in food delivery, grocery, and rideshare categories in the US and Korea. This allows Gemini to, for instance, coordinate complex pizza orders or multi-stop rideshares by interacting with the app's UI as a human user would, but autonomously.

Both capabilities are in early beta stages, with plans to broaden their availability and functionality in Android 17, as Google continues to refine these experiences in collaboration with a small set of app developers.