DocumentationExtend the framework
Extend clients and generated UI
Share behavior through contracts while keeping each platform's permissions and presentation native.
On this page
Platform boundariesChoose the right surfaceGenerated UI: from request to renderInteractive HTMLNative adaptersImplementation and checksiOS, Android, Web and Mac speak the Impo API; the Apple Watch app reaches it through the paired iPhone and uploads Rewind parts directly. SwiftUI and Compose provide the mobile interfaces. Web uses React; Mac embeds that interface in Electron and adds native computer-access adapters. Shared contracts define command identity, ownership, events and recovery.
Platform boundaries#
| Platform | Capabilities specific to that client |
|---|---|
| iOS | Native phone tools, Rewind recording, Bluetooth gadget setup and dedicated Gadget Conversation/Device pages. |
| Apple Watch | Ships with the iOS app. Ask Impo by voice into Main Chat through the paired iPhone, and record Rewind on the watch. Ask Impo and Rewind complications. No chat history or text input on the watch. |
| Android | Native phone tools, Rewind recording and Bluetooth gadget setup. Navigation still uses the earlier Task labels. |
| Web | Browser conversations, generated UI, connected services and Rewind history. Dictation requires a browser microphone grant. No background Rewind, native phone tools or Bluetooth gadget setup. |
| Mac | The shared Web interface plus explicitly enabled file, command, screen and input capabilities. It uses native-device registration, not the Gadget SDK. Public distribution is currently Apple Silicon macOS. |
Hardware commands execute on a paired gadget through server tools; they are separate from a client's native capabilities. A Web conversation can therefore ask a connected gadget to take a photo without giving the browser native camera access.
Choose the right surface#
| Change | Extension path |
|---|---|
| A permanent navigation screen | Add an application API contract and a platform screen with loading, empty, error and restored states. |
| A one-off form or visual response | Write a widget block in the answer with the supported A2UI catalog; the server compiles it in place. |
| A new native capability | Define a device tool and implement an adapter on each supporting platform. |
| An action that requires the user's tap | Prepare an owned client action and dispatch only from the foreground interaction. |
Generated UI: from request to render#
The user sends an ordinary Chat or Task message. The agent receives a short widget authoring guide and reads only the needed component schemas through a documentation tool. It writes widget blocks inside its answer; the worker validates and compiles each block in place before clients render it.
iOS and Android render the catalog with AGenUI. Web and Mac use React and Astryx for the same impo.mobile.v1 contract. Button actions become ordinary follow-up messages through the client's durable send path. The UI document is not executable authority.
To add a catalog component, update the server validator, shared contract, all supporting renderers and the summary fallback. Keep installed clients that do not understand the new component usable.
Interactive HTML#
Offline HTML games use an opaque sandbox with no host bridge. They cannot read the account, access provider keys or invoke native tools. Web adds an enclosing frame to restrict navigation.
Generated UI is suitable for bounded forms and visual interaction. A persistent application feature with durable local state needs an explicit product contract. Unsent generated-form state can reset when a card unmounts or reloads.
Native adapters#
Register capabilities only while available. A device claims owned pending work before executing it, persists the result before uploading it, and reuses the accepted receipt after a retry. Stream replay never authorizes a second native action.
Keep permission prompts as explicit UI actions. Do not translate empty health data into a denial, or use one phone's grant to act on another. Web does not implement native Health, contacts, background Rewind or BLE gadget setup.
Implementation and checks#
- Protocol models:
contracts/,ios/Packages/InstantClient,android/client, andweb/src/. - Native presentation and adapters:
ios/App/Sources/andandroid/app/src/main/java/ai/impo/. - Web presentation:
web/src/, with the existing Astryx theme and components. - Desktop host and native adapters:
desktop/; see the Desktop guide for grants and local execution limits.
Run the relevant root commands: test:swift, test:android, test:web, test:desktop, and the affected UI checks. Verify recovery, account switching, narrow layouts, keyboard access and dark appearance. Regenerate the Xcode project with npm run generate:ios when adding native files.
The generated UI contract and capability contract define supported inputs and fallbacks.