Hermes iOS Companion

Overview

Hermes iOS Companion is an independently built mobile control surface for the open-source Hermes Agent. It lets me continue conversations, inspect tool execution, handle security approvals, use voice input, and follow long-running tasks when I am away from the computer that hosts the agent.

The project is a deployed personal system and private beta, not a general consumer chat app. I designed and built the iOS client, its interaction model, gateway protocol integration, security boundaries, and Apple platform surfaces. It is not an official Nous Research product.

The Problem: Desktop Agents Lose Their Interface When You Walk Away

A desktop AI agent can read files, run tools, and complete long tasks, but its interface usually disappears the moment the user leaves the desk. A mobile chat box alone does not solve that problem. The phone also needs to communicate execution state, preserve session continuity, surface system events, and make high-risk actions explicit.

The product goal was therefore not to shrink the desktop experience. It was to identify the few responsibilities that matter on a phone: lightweight input, status awareness, interruption handling, and trusted confirmation.

The Product Decision: Keep the Runtime Off the Phone

The iPhone does not run the agent runtime. Hermes remains on a Mac or VPS, where it has the files, tools, compute, and long-lived process context it needs. The phone acts as a secure control surface that sends input and renders state from the Gateway.

This boundary reduced product and security complexity. The Gateway remains the source of truth. The mobile client can reconnect, reconcile, or lose a transient event without becoming a second authority for the task itself.

Architecture: One Source of Truth, Several Feedback Paths

The iPhone reaches each Hermes Gateway through Tailscale HTTPS. Mac and VPS profiles keep endpoint identity, credentials, sessions, and notification routing separate. Requests move to the selected Gateway, while SSE streams provide foreground progress and APNs, Widget, Live Activity, and Dynamic Island provide system-level status.

SSE and push are feedback layers, not competing databases. After interruption or reconnect, the client reconciles against Gateway state instead of assuming every transient event was received.

Core Challenges

The difficult work was not rendering chat bubbles. It was preserving trust and continuity across networks, processes, devices, and iOS lifecycle boundaries.

Remote Control Must Stay Legible

Remote execution creates a human factors problem: the user needs enough context to make a decision without reading a desktop-sized trace on a phone. The approval flow exposes intent and authorization scope before allowing the action. For high-risk requests, Face ID or device passcode confirms that the person holding the phone is the person granting access.

During execution, reasoning and tool calls collapse into a compact progress surface. Details remain available for inspection, but the final answer stays outside the trace so status, diagnostic detail, and outcome do not compete for attention. The screens shown here use safe simulator fixtures from the current main build.

Validation Snapshot

The strongest evidence is the complete system running across a physical iPhone and two self-hosted environments. Automated tests cover deterministic behavior, while device acceptance verifies the Apple services and network conditions that mocks cannot prove.

Known Limits and Release Boundary

Hermes iOS Companion is a private beta and deployed personal system. It is not on the App Store, is not publicly available, and is not presented as an official Hermes app. Setup still assumes a self-hosted Hermes Gateway and Tailscale access, which is appropriate for the current audience but not consumer onboarding.

Public screenshots use example data and sanitized fixtures. Live Gateway URLs, API keys, pairing payloads, Tailscale domains, device tokens, APNs credentials, and private conversation content are intentionally excluded.