Canvas A2UI Integration Patterns

Introduction

Integrating A2UI with OpenClaw's Canvas system on macOS creates a powerful, dynamic interface for agent interactions. This guide outlines proven patterns for effective A2UI implementation within the Canvas environment.

Core Architecture

The integration operates on a client-server model:

• Client: Canvas panel in the OpenClaw macOS app (WKWebView)
• Server: Gateway-hosted A2UI renderer
• Protocol: WebSocket-based communication using A2UI v0.8 messages

The Canvas automatically navigates to the A2UI host when available:

http://<gateway-host>:18789/__openclaw__/a2ui/

Supported Message Types

Canvas currently supports A2UI v0.8 server→client messages:

• beginRendering - Initialize surface rendering
• surfaceUpdate - Update component structure
• dataModelUpdate - Modify data model
• deleteSurface - Remove a surface

Note: createSurface (v0.9) is not yet supported.

Implementation Patterns

1. Progressive Rendering Pattern

For complex UIs, use progressive rendering to improve perceived performance:

{"surfaceUpdate":{"surfaceId":"main","components":[{"id":"root","component":{"Column":{"children":{"explicitList":["header","content","footer"]}}}},{"id":"header","component":{"Text":{"text":{"literalString":"Loading..."},"usageHint":"h1"}}},{"id":"content","component":{"Text":{"text":{"literalString":"Initializing interface"},"usageHint":"body"}}},{"id":"footer","component":{"Text":{"text":{"literalString":"Please wait"},"usageHint":"small"}}}]}}
{"beginRendering":{"surfaceId":"main","root":"root"}}

Follow with subsequent updates as data becomes available.

2. Data-Driven UI Pattern

Structure your A2UI messages to separate presentation from data:

// Initial structure
{"surfaceUpdate":{"surfaceId":"dashboard","components":[{"id":"container","component":{"Grid":{"columns":2,"rows":2,"children":{"explicitList":["cpu","memory","network","disk"]}}}},{"id":"cpu","component":{"Card":{"title":{"literalString":"CPU"},"contentId":"cpu_data"}}},{"id":"memory","component":{"Card":{"title":{"literalString":"Memory"},"contentId":"memory_data"}}},{"id":"network","component":{"Card":{"title":{"literalString":"Network"},"contentId":"network_data"}}},{"id":"disk","component":{"Card":{"title":{"literalString":"Disk"},"contentId":"disk_data"}}}]}}

// Data updates
{"dataModelUpdate":{"dataModel":{"cpu_data":{"component":{"Gauge":{"value":75,"label":{"literalString":"75%"}}}},"memory_data":{"component":{"Gauge":{"value":82,"label":{"literalString":"82%"}}}},"network_data":{"component":{"Gauge":{"value":45,"label":{"literalString":"45%"}}}},"disk_data":{"component":{"Gauge":{"value":68,"label":{"literalString":"68%"}}}}}}}

3. Interactive Command Pattern

Enable user-triggered actions through Canvas deep links:

window.location.href = "openclaw://agent?message=Refresh%20dashboard%20data";

File System Structure

Canvas state is stored under Application Support:

~/Library/Application Support/OpenClaw/canvas/<session>/...

Access files via the custom URL scheme:

openclaw-canvas://<session>/<path>

CLI Operations

Use these commands to control Canvas:

# Show Canvas panel
openclaw nodes canvas present --node <id>

# Navigate to path
openclaw nodes canvas navigate --node <id> --url "/"

# Execute JavaScript
openclaw nodes canvas eval --node <id> --js "document.title"

# Capture image
openclaw nodes canvas snapshot --node <id>

# Push A2UI messages
openclaw nodes canvas a2ui push --jsonl /path/to/messages.jsonl --node <id>

Security Considerations

• Canvas scheme blocks directory traversal
• Local content uses custom scheme (no loopback server)
• External URLs allowed only when explicitly navigated
• Panel is borderless and anchored near menu bar or cursor

Troubleshooting

• If Canvas doesn't auto-navigate to A2UI, ensure the Gateway is advertising the Canvas host
• For message delivery issues, verify WebSocket connection between Gateway and macOS app
• Check that A2UI messages use v0.8 format (v0.9 createSurface not supported)
• Ensure the session root contains appropriate files for requested paths

The Canvas panel remembers size and position per session and auto-reloads when local files change.