Skip to content

Protocols

The wire formats Gopala’s parts speak: guest agent and host over vsock, the LLM, MCP, egress and git proxies, watch rooms, monitoring and the HTTP API.

Message names refer to Gopala’s protobuf schema; constants such as VSOCK_CONTROL_PORT are the protocol’s named values. For how the pieces fit together see architecture.md.

Protocol Between Transport Encoding
Control channel guest uplink ⇄ host session vsock, guest → host CID 2 port 1024 protobuf GuestFrame / HostFrame, u32 length prefix
MCP the harness ⇄ gopala-guest mcp ⇄ host ScopedMcp stdio in the VM, vsock port 1025 to the host MCP JSON-RPC (rmcp)
omp RPC gopala-guest ⇄ omp omp’s stdin/stdout newline-delimited JSON
Claude Code stream-json gopala-guest ⇄ claude -p Claude Code’s stdin/stdout newline-delimited JSON
Viewer websocket gopala watch (or any viewer) ⇄ Rooms ws / wss at /watch on the [server] protobuf ClientFrame / ServerFrame, one per binary message
Monitor websocket any app (the Lupyd app, scripts) ⇄ the host’s monitor ws / wss at /monitor on the [server] protobuf MonitorClientMessage / MonitorServerMessage, one per binary message; the same answers over HTTP under /api/monitor/
LLM proxy harness ⇄ guest forwarder ⇄ host llm_proxy ⇄ provider TCP 127.0.0.1 in the VM, vsock port 1100+i, HTTPS out HTTP/1.1
Egress proxy programs in the VM ⇄ guest forwarder ⇄ host egress ⇄ allowed hosts http_proxy=http://127.0.0.1:3128 in the VM, vsock port 1090, TCP out HTTP/1.1 proxy requests (CONNECT, absolute-form http://)
HTTP MCP proxies the harness ⇄ guest forwarder ⇄ host ⇄ an operator-configured MCP server 127.0.0.1:3200+i in the VM, vsock port 1200+i, HTTP(S) out MCP streamable HTTP
Git proxy git, curl ⇄ guest forwarder ⇄ host git proxy ⇄ GitHub, GitLab or Gitea 127.0.0.1:3300+i in the VM, vsock port 1300+i, HTTPS out git smart HTTP, the provider’s REST API
HTTP API dashboard, /configure/, curl ⇄ the host’s web server HTTP(S) on server.bind JSON, TOML, server-sent events
Chat platforms provider ⇄ its platform Lupyd: Firefly over WebTransport / WebSocket; Slack: Socket Mode WebSocket + Web API; Discord: gateway WebSocket + REST the platform’s own, hidden behind the Platform trait; see providers.md

All connections are opened by the guest (AF_VSOCK, see vsock(7)) to the host CID (HOST_CID = 2). Firecracker turns a guest connection to port P into a connection on the host Unix socket {uds_path}_{P} (Firecracker vsock docs, “guest-initiated connections”). The host binds exactly these, per VM, before booting it (session::run_vm via PreparedVm::bind_vsock):

Port Constant Purpose Host listener
1024 VSOCK_CONTROL_PORT control channel {uds}_1024, accepted by session::handshake and the reconnect arm of run_vm
1025 VSOCK_MCP_PORT (also HostInit.mcp_vsock_port) one MCP session per connection session::serve_mcp
1090 VSOCK_EGRESS_PORT egress proxy, only with vm.network = "proxy" egress::serve
1100 + i VSOCK_LLM_BASE_PORT + index in [[llm]] model API proxy i llm_proxy::serve
1200 + i VSOCK_MCP_HTTP_BASE_PORT + index in agent.mcp proxy for HTTP MCP server i (only entries with a url) egress::serve_mcp
1300 + i VSOCK_GIT_BASE_PORT + index in [[git]] git proxy for account i the host’s git proxy

Inside the guest the forwarders listen on 127.0.0.1: [[llm]].guest_port for the model proxies, GUEST_EGRESS_PORT (3128) for the egress proxy, GUEST_MCP_HTTP_BASE_PORT + i (3200 + i) for HTTP MCP servers and GUEST_GIT_BASE_PORT + i (3300 + i) for [[git]] accounts. Config::validate keeps guest_port off 3128, off 3200..3200+n, n being the number of agent.mcp entries, and off 3300..3300+m, m being the number of [[git]] entries.

The guest’s CID is 3 + slot. Host sockets are mode 0600 (chowned to the jailer uid/gid when jailed) and the full path must fit sun_path (107 bytes; bind_vsock errors otherwise). Nothing listens on other ports, so a guest connection to any other port fails.

gopala_proto::codec(): tokio-util’s LengthDelimitedCodec with a 4-byte big-endian length followed by the protobuf message; frames longer than MAX_FRAME (1 MiB = 1 048 576 bytes) are a decode error. The guest sends GuestFrames, the host HostFrames, each a oneof frame.

sequenceDiagram
    participant G as guest (uplink::session)
    participant H as host (session::handshake / run_vm)
    G->>H: connect vsock 2:1024
    G->>H: GuestFrame.hello{protocol: 1, version}
    Note over H: within Timing::hello (10 s) of accept, and the whole<br/>boot within vm.boot_timeout_secs
    H->>G: HostFrame.init(HostInit)
    par guest → host, any order
      G->>H: event(SessionEvent)*
      G->>H: status(GuestStatus)*
      G->>H: approval(ApprovalRequest)*
      G->>H: usage(ResourceUsage) every usage_interval_secs
    and host → guest, any order
      H->>G: deliver(Delivery)*
      H->>G: approval(ApprovalResult)*
      H->>G: abort(Abort)*
    end
    H->>G: shutdown(Shutdown)
    Note over G: stop harness, sync, reboot → Firecracker exits

The host rejects a first frame that isn’t hello (“the guest didn’t introduce itself”) or whose protocol differs from CONTROL_PROTOCOL (“guest speaks control protocol X, host Y”); both end the boot attempt with an error and a retry with backoff. Unknown or empty oneof values are ignored on both sides; a frame that fails to decode ends the host’s reader (spawn_reader) and is treated as a disconnect.

Variant Message Fields Guest behaviour
init (1) HostInit scope (1): the conversation key (ConversationId::key, e.g. lupyd-g-42, lupyd-d-<16 hex>); title (2); bot_username (3); harness (4): Harness{kind: HARNESS_OMP | HARNESS_EXEC | HARNESS_CLAUDE_CODE, command, args}; packages (5); setup (6): root shell commands; privileges (7): Privileges{sudo, install_packages}; llm (8): LlmForward{local_port, vsock_port}; env (9): EnvVar{key, value} (agent.env then every [[llm]].guest_env); system_prompt (10): built by policy::system_prompt; mcp_vsock_port (11): 1025; user_setup (12): agent-user shell commands; disk_alert_percent (13), memory_alert_percent (14): agent.*_alert_percent; usage_interval_secs (15): agent.usage_interval_secs; tools (16): agent.tools; mcp (17): McpServer{name, command, args, env, url} per agent.mcp entry, url rewritten to http://127.0.0.1:3200+i/<path> for HTTP servers; egress_proxy (18): LlmForward{3128, 1090} with vm.network = "proxy", unset otherwise; mcp_forward (19): LlmForward{3200+i, 1200+i} per HTTP MCP server; memory (20): MemoryConfig with the [memory] limits (max_memories, max_db_mib, half_life_days, archive_days, duplicate_threshold, cleanup_interval_secs), embeddings_port/_model/_dimensions and compaction_port/_model/_format/_every_hours/_min_memories/_max_clusters, where each port is the named [[llm]] entry’s guest_port (0 = off); unset when memory.enabled = false (session::memory_config); git (21): GitForward{name, provider, url, local_port, vsock_port} per [[git]] entry, with provider a GitProvider (GitHub, GitLab, Gitea), url the configured base URL (no credentials), ports 3300+i and 1300+i First one: start the egress forwarder and write the proxy settings (proxy::install), provision (through the proxy), start the model, MCP and git forwarders, write the git URL rewrites (/etc/gopala-git.conf, included from /etc/gitconfig) and GOPALA_GIT_<NAME>_API, write the system prompt, start the resource monitor, the memory maintenance loop (memory::spawn_maintenance, with memory set) and the harness. Before every harness start the guest writes its MCP config: the host’s servers (agent_config::base) merged with the agent’s own /home/agent/.gopala/mcp.json (agent_config::merge, see architecture.md). agent.tools becomes --tools=a,b for omp and Claude Code (agent_config::tool_args); omp gets --no-tools when none of the names is valid, Claude Code --tools=. Repeats (after a reconnect) are ignored by a running guest process. git was added without changing CONTROL_PROTOCOL; an older guest ignores it.
deliver (2) Delivery text (1): the formatted chat message (see Delivery text); urgent (2): steer instead of follow-up (the host always sends false; the guest’s own resource alerts are urgent) Queued until the harness is ready; omp: prompt / follow_up / steer; Claude Code: a user message on stdin, with "priority": "later" while a turn runs unless urgent; exec: one run. Queued deliveries survive a harness restart.
approval (3) ApprovalResult id (1): the request’s id; choice (2): option index, for yes/no 0 = yes and 1 = no, -1 = no answer; answered_by (3); timed_out (4): set whenever choice < 0 Omp::on_approval → extension_ui_response; unknown ids ignored. Claude Code never asks (it runs with bypassPermissions), so it ignores these.
abort (4) Abort — omp: queued deliveries cleared, abort command; Claude Code: queued deliveries cleared and, if a turn runs, the process stopped and started again with --continue; exec: the running process is killed. The host also clears its own queue.
shutdown (5) Shutdown — Harness stopped (stdin closed, killed after 10 s), sync, reboot(RB_AUTOBOOT); with reboot=k on the kernel command line Firecracker exits. Before HostInit it powers off immediately.
restart_harness (6) RestartHarness reason (1): the agent’s reason, at most 300 characters Sent by the host about 1 s after the agent’s restart_harness tool call. omp and Claude Code: lifecycle restarting (“restarting harness: ”), SIGTERM, a kill after 10 s, then a new run of the same session (-c / --continue) at once, without backoff and without a crashed event; its first prompt tells the agent the harness restarted, why, and which of its own MCP servers loaded or were left out. exec: a notice that there’s nothing to restart, and .mcp.json rewritten.
Variant Message Fields Host behaviour
hello (1) GuestHello protocol (1): CONTROL_PROTOCOL = 1; version (2): the crate version Checked on the first connection; on reconnect only its presence is checked. Ignored later.
event (2) SessionEvent seq (1): 0 from the guest (the host overwrites it); at_ms (2): Unix ms (set by the host if 0); kind (3): EventKind; actor (4): main or a subagent id (host for host-made events); title (5); text (6); is_error (7) Resets the idle timer, then HostConversation::journal_event → Rooms::publish.
approval (3) ApprovalRequest id (1): unique per guest process; question (2); options (3): empty = yes/no; timeout_secs (4): 0 = the host’s limit Spawned request_approval: options cut to 10, a single option becomes yes/no, timeout min(timeout_secs, approvals.timeout_secs); the ApprovalResult comes back on this channel.
status (4) GuestStatus state (1): provisioning | ready | busy | idle | failed; detail (2) busy/provisioning keep the VM from idling out; failed is logged; /status shows working or idle.
usage (5) ResourceUsage mem_total_kib (1), mem_available_kib (2) from /proc/meminfo; disk_total_kib (3), disk_free_kib (4): statvfs("/"), free = what an unprivileged user may still use; load_milli (5): 1-minute load average × 1000; vcpus (6): online CPUs VmControl::record_usage keeps the latest sample for vm_resources and /status. Not journaled, and doesn’t reset the idle timer.

policy::format_delivery renders each message that wakes the agent as one header line and the text:

alice in #general (channel 1) [message 1718000000000001, replying to 1717999999000000]:
@bot can you look at the failing build?

The channel part appears only when the platform has channels (not in direct chats, never on Slack), replying to only for replies, and a line about attached files when there are any (the files themselves aren’t available). A message from another agent (Incoming.from_agent) is marked after the sender’s name, so the agent can tell a ping from a person’s request:

researcher (agent) in #general (channel 1) [message 1718000000000002]:
@bot the three papers you asked about are summarized in the thread

EventKind values and their conventions:

Kind title / text Produced by
EVENT_INPUT (1) prompt / follow_up / steer (omp and Claude Code) or prompt (exec); text = the delivery (including resource alerts) guest Omp::deliver, Claude::deliver, exec_loop
EVENT_TEXT (2), EVENT_THINKING (3) — / coalesced deltas guest Coalesce
EVENT_TOOL_CALL (4) tool name / arguments JSON omp tool_execution_start; Claude Code tool_use blocks
EVENT_TOOL_RESULT (5) tool name / result text, is_error omp tool_execution_end; Claude Code tool_result blocks
EVENT_LIFECYCLE (6) booting provisioning ready busy idle restarting refused stopped crashed guest and host (session::run)
EVENT_SUBAGENT (7) actor = subagent id; title = status or agent; text = task / progress omp subagent_lifecycle, subagent_progress; Claude Code: a subagent’s first prompt (title prompt, actor = the parent tool call’s id)
EVENT_NOTICE (8) level, retry, compaction, rpc_chunk, rpc_frame_error, failed command name, exit; Claude Code: mcp (a server that didn’t connect), error, or the result subtype of a failed turn; resources (guest alert, is_error; host grants, denials, restarts, sleeps); harness (the agent asked for a harness restart, host); network (egress proxy allowed/refused, host); budget (model calls refused, host); listen (someone switched listening mode with /listen, host); operator (dashboard actions) guest; host VmControl::journal, egress journal, budget::gate, Router, Control
EVENT_LOG (9) step name (useradd, packages, setup[i], user_setup[i], provisioning), init (Claude Code’s version and model) or stderr guest provisioning, Claude Code’s system/init, and harness stderr
EVENT_APPROVAL (10) asked / answered / timed_out host HostConversation::request_approval
EVENT_OUTBOUND (11) message, message (warning), message (critical) / edit / delete, text sent host HostConversation::say, send, host_event, edit, delete
EVENT_LLM (12) title = upstream name and model; text = status, duration, input/output tokens and cache reads/writes; is_error for non-2xx, refusals and upstream failures host, one per model call through the LLM proxy (session::llm_meter)
  • Guest (uplink::run): reconnects forever with backoff 200 ms doubling to 5 s (reset on a successful connect), sends a fresh GuestHello each time. Outgoing frames sit in the Outbox (QUEUE_CAP = 2000 frames, oldest dropped when full); a frame whose write fails is put back at the front. So the harness never blocks on the host.
  • Host (run_vm): when the reader ends, the session marks the guest disconnected but keeps the VM. Deliveries stay queued (a delivery whose write failed is pushed back to the front), ApprovalResults are held in unsent_answers, and the idle timer and VM exit are still watched. The next connection on {uds}_1024 must send hello within 10 s; the host then resends HostInit, the held answers and the queue.
  • A running guest process ignores a repeated HostInit. A restarted guest process (systemd Restart=always) treats it as a first boot: provisioning is skipped if the marker hash matches, and the harness continues its session (omp with -c, Claude Code with --continue).
  • Frames already written into the socket when a connection broke may be lost (the host only requeues deliveries whose write returned an error) [inference from the code].
Limit Value Where
Control frame 1 MiB MAX_FRAME, both sides; the guest drops larger frames before sending (Outbox::send)
Event text from the guest 16 KiB uplink::EVENT_TEXT_MAX (gopala_proto::truncate, marks the cut with …)
Event stored in the journal MAX_FRAME - 1024; above it actor → 256 B, title → 4 KiB, text → half Rooms::publish
Host-made event text 16 KiB (journal), 4 KiB (crash text) HostConversation::journal, session::run
Guest outbox 2000 frames uplink::QUEUE_CAP
Host reader channel 512 frames (back-pressure on the socket) spawn_reader
Text/thinking coalescing flushed at 4 KiB, on a 500 ms tick, on *_end (Claude Code: after every content block), and before any other frame omp::COALESCE_MAX, harness::run_session
Harness stderr 50 lines/s (excess counted and reported), lines ≤ 2 KiB harness::spawn_log_pump
Provisioning output lines ≤ 2 KiB provision::LOG_LINE_MAX
Over-long lines dropped and replaced by (line over 2048 bytes dropped); omp stdout lines over 1 MiB + 1 KiB and Claude Code stdout lines over 8 MiB are discarded sys::Lines, harness::OMP_LINE_MAX, harness::CLAUDE_LINE_MAX
The agent’s own MCP file 64 KiB, 32 servers; a larger file is ignored whole, servers past 32 are left out agent_config::AGENT_MCP_MAX, AGENT_SERVERS_MAX
Approval options ≤ 10 (host truncates), question ≤ 16 KiB from omp run_vm, Omp::ui_request
Usage reports every usage_interval_secs (≥ 5 s; 30 s if 0); an unreadable sample is skipped monitor::run

The harness starts its MCP servers from a config gopala-guest writes before every start (agent_config::base, then agent_config::merge): always chat, the host’s server; memory, the built-in memory server, when HostInit.memory is set; every valid agent.mcp entry, stdio (command, args, env, run inside the VM as the agent user) or streamable HTTP ("type": "http", url on the guest’s forwarder); and the servers the agent added to its own /home/agent/.gopala/mcp.json:

{"mcpServers": {
"chat": {"type": "stdio", "command": "/usr/local/bin/gopala-guest", "args": ["mcp", "--port", "1025"]},
"memory": {"type": "stdio", "command": "/usr/local/bin/gopala-guest",
"args": ["memory", "--db", "/home/agent/.gopala/memory.db", "--max-memories", "5000", "..."]},
"fetch": {"type": "stdio", "command": "uvx", "args": ["mcp-server-fetch"]},
"tracker": {"type": "http", "url": "http://127.0.0.1:3200/mcp"}
}}

The guest checks the operator’s entries again and leaves out (logged) any that’s malformed, named chat, named memory while the built-in one is on, or listed twice. The agent’s file takes the same entry shapes ({"command", "args"?, "env"?} or {"url"}, with an optional "type" of stdio or http; other keys are ignored) and can’t use chat, memory or any operator server’s name; the rules are in architecture.md. omp and exec read the result from /home/agent/.omp/agent/mcp.json and /home/agent/work/.mcp.json, Claude Code from /home/agent/.gopala/claude-mcp.json (--mcp-config, with --strict-mcp-config).

gopala-guest mcp connects to vsock 2:1025 and copies bytes in both directions between its stdin/stdout and the socket; it exits when either side closes. On the host each connection is one rmcp server session (ScopedMcp::serve) over the socket’s byte stream (newline-delimited JSON-RPC, as in MCP’s stdio transport). The server advertises only the tools capability, server info gopala <version>, and instructions. Tool errors are returned as MCP tool errors with a plain message; results are JSON objects.

ScopedMcp is constructed with one Arc<dyn Conversation>, the HostConversation of the VM whose socket the connection arrived on, and that VM’s Machine (VmControl, via ScopedMcp::with_machine in session::serve_mcp). No tool takes a platform, a conversation id, a username or a scope. Ids that a tool does take (message_id, before_id, channel_id) are resolved inside that conversation by its platform provider: channel_id must be one of the conversation’s channels (HostConversation::channel), and edit/delete check that the message exists in this conversation and was sent by the bot (HostConversation::check_own, via Platform::message). Channel ids are opaque strings (Lupyd uses the channel number).

Tool Parameters Result Limits and behaviour
conversation_info none ConversationInfo{kind: "group" | "direct", title, description?, you, members, channels?} Cached metadata, refreshed on member/group changes.
read_messages channel_id?, before_id?, limit? Page{messages (oldest first), older_before_id?} limit default 30, clamped 1–100; Platform::history with edits, deletes and reactions applied. Lupyd reads up to 3 × limit (≤ 100) raw rows and folds them (firefly_mcp::content::fold); hidden DM rows are skipped; reading the newest DM page marks it read.
send_message text, channel_id?, reply_to?, severity? (info default, warning, critical) Sent{message_id, note?} (message_id of the last part) Empty text refused. Split into parts of ≤ behavior.max_message_chars characters (line breaks, then spaces), each part counted against behavior.max_messages_per_minute; over the limit the tool errors with a rate-limit message. Without channel_id: the channel the conversation last happened in, else the platform’s default (Lupyd: the first channel). A warning or critical message notifies the audience [alerts] sets for this conversation (first part only, through Platform::send_notifying); the agent never names people or mention syntax. In a direct chat, or once alerts.max_per_hour notifying messages went out this hour, it is sent without notifying anyone and note says so. Journaled as EVENT_OUTBOUND titled message, or message (warning) / message (critical). See Alerts.
edit_message message_id, text Sent Only the bot’s own messages (HostConversation::check_own).
delete_message message_id Sent As above.
react message_id, emoji (empty = remove) Sent Any message in this conversation.
wait_for_events after_seq?, timeout_seconds?, limit? Events{events: ChatEvent[], next_after_seq} timeout_seconds default 60, max 600; limit default 50, clamped 1–500. A cursor ahead of the log (host restarted) restarts from 0. Events: message, edit, delete, reaction, members_changed, updated, each with seq and at. The log keeps the last 1000 events in memory.
search_messages query, limit? SearchResults{results: SearchHit[]} limit default 20, clamped 1–100; the platform’s search restricted to this conversation (Lupyd: Firefly’s local full-text index).
request_approval question, options?, timeout_seconds? ApprovalOutcome{approved, choice?, choice_text?, answered_by?, timed_out?} Empty question refused; options must be absent/empty (yes/no) or 2–10; timeout default and maximum approvals.timeout_secs; at most 3 open per conversation, and each question counts against behavior.max_messages_per_minute. Blocks until answered or timed out. See architecture.md.
vm_resources none VmReport{allocated, after_restart?, maximum, usage?, grants} allocated: what the VM runs with (Allocation{vcpus, memory_mib, disk_gib}); after_restart: a grant not applied yet; maximum: vm.max_vcpus, min(vm.max_memory_mib, memory budget), vm.max_disk_gib; usage: {memory_used_percent, memory_available_mib, disk_used_percent, disk_free_mib, load, age_seconds} from the last ResourceUsage; grants: auto, approval or never.
request_resources vcpus?, memory_mib?, disk_gib? (totals, not increments), reason Decision{granted, after_restart, message} At least one value and a non-empty reason. Decided by machine::decide (see architecture.md); with vm.grants = "approval" growth goes through request_approval first. A grant is persisted and applies at the next restart.
restart_vm reason Done{message} Journals the reason; about 1 s later the session stops the VM and boots it again with the desired allocation, without waiting for a message.
sleep_vm reason Done{message} Journals the reason; about 1 s later the VM stops as if idle. The next delivery wakes it (queued deliveries boot it again right away).
restart_harness reason Done{message}: “restarting your harness in a moment; this session continues, and a message tells you when you’re back” Restarts the harness inside the running VM, for example to load a server the agent added to its MCP file. The reason is trimmed, must not be empty (“give a reason”) and is cut to 300 characters. The host journals a harness notice and about 1 s later sends HostFrame.restart_harness, so the tool result arrives first; the VM keeps running and queued deliveries wait for the new run. With agent.harness = "exec" it errors: “your harness starts once per message, so there is nothing to restart”.
list_agents none AgentList{agents: AgentInfo[], note}, AgentInfo{username, display_name?, about?, roles, bot} The other agents in this group, from Platform::agents with collaboration.agent_role (see providers.md): at most 50, each about cut to 2000 characters, display_name and about left out when empty. note says “mention them by username to ping; they decide for themselves”. In a direct chat: no agents and the note “no other agents in a direct chat”, without asking the platform. Cached per conversation for 5 minutes and dropped on MembersChanged and Updated. Errors: “collaboration is off on this host” (collaboration.enabled = false), “couldn’t list agents: ” when the platform fails (the full error goes to the log). Counted as a tool call.

The five VM tools (vm_resources, request_resources, restart_vm, sleep_vm, restart_harness) return the error VM controls aren't available here when the server was built without a Machine (only in tests).

Message ids are the platform’s (u64, unique within a conversation; Lupyd: server time in microseconds, as INSTRUCTIONS tells the agent).

memory is a second MCP server, built into gopala-guest and served inside the VM by gopala-guest memory --db /home/agent/.gopala/memory.db … (gopala_memory::server::MemoryServer). omp starts it on stdio as the agent user, so it never talks to the host except for embeddings, which go through an [[llm]] forwarder. Several processes can open the database at once (omp subagents start their own server): SQLite WAL, a 10 s busy timeout and short IMMEDIATE write transactions. Server info memory <version>, with instructions that tell the agent what’s worth keeping.

Tool Parameters Result Limits and behaviour
remember text, importance? (0–1, default 0.5), entities? ([{name, kind?}]), relations? ([{from, relation, to}]), pinned? {id, merged_into?} Text ≤ 2 KiB after whitespace normalization; names ≤ 128 B; ≤ 16 entities and ≤ 16 relations. An exact duplicate (same normalized text, case-insensitive) or a near duplicate (word-set Dice over the 10 best BM25 matches, or cosine with embeddings, at or above duplicate_threshold) merges into the existing memory.
recall query, limit? (default 10, ≤ 50), entity? {memories: [{id, text, importance, pinned, entities, updated, score}]} Reciprocal rank fusion (k = 60) of FTS5 BM25, embedding cosine (when on) and memories of entities the query names, then × 1.5 for an entity match, × (0.5 + importance), × (0.5 + 0.5 × recency). Archived memories are left out. Counts as a use of each hit.
update id, text?, importance?, pinned? the memory Restores an archived memory.
forget id {message} Deletes the memory and the relations it stated.
relate / unrelate from, relation, to the relation / {message} Relations between entities, unique per (from, relation, to).
entity name (case-insensitive) its memories (newest first) and relations both ways
stats none counts, size, limits, last cleanup and compaction

A tool call waits at most 8 s for an embedding (EMBED_WAIT) and goes on without one; the maintenance pass fills in what’s missing. Embeddings are POST /v1/embeddings (OpenAI format) and compaction POST /v1/messages (Anthropic) or /v1/chat/completions (OpenAI) to 127.0.0.1:<guest_port>, so the host’s LLM proxy adds the key, meters the tokens and applies the budget like any other model call. Cleanup and compaction are described in architecture.md.

The guest drives omp’s RPC mode over the child’s stdin (JSON commands, one per line) and stdout (JSON frames, one per line). The protocol handling is a pure state machine (Omp); process plumbing is harness::run_session, shared with Claude Code.

Command line: omp --mode rpc <agent.args> --append-system-prompt /home/agent/.gopala/system.md plus -c after the first successful start. With the default agent.args that is --approval-mode yolo, so omp doesn’t stop for its own tool approvals.

Each command gets "id": "g<n>" (increasing per process) and "type".

When Command
On ready {"type":"negotiate_protocol","protocolVersion":2} then {"type":"set_subagent_subscription","level":"progress"}
Delivery while idle {"type":"prompt","message":…}
Delivery while busy {"type":"follow_up","message":…}, or {"type":"steer",…} when Delivery.urgent
Abort {"type":"abort"}
Approval answered {"type":"extension_ui_response","id":<ui id>, …} with "confirmed": bool (confirm), "value": <option> (select), or "cancelled": true (no answer, invalid choice), plus "timedOut": true on timeout
input / editor UI request {"type":"extension_ui_response","id":…,"cancelled":true} immediately

Deliveries that arrive before ready (or while omp restarts) are kept in order and sent on ready (harness::apply, Action::Ready).

omp frame Mapping
ready EVENT_LIFECYCLE ready, GuestStatus ready; writes /home/agent/.gopala/omp-started
message_update with assistantMessageEvent.type text_delta / thinking_delta Coalesced into EVENT_TEXT / EVENT_THINKING; text_end / thinking_end flush. A coalesced buffer over 16 KiB is split on char boundaries into several events rather than truncated.
tool_execution_start EVENT_TOOL_CALL, title toolName, text args as JSON
tool_execution_end EVENT_TOOL_RESULT, title toolName, text = the content[].text blocks joined (or the raw value), is_error = isError
agent_start EVENT_LIFECYCLE busy, GuestStatus busy
agent_end (unless isTerminal: false), prompt_result with agentInvoked: false, failed response to prompt EVENT_LIFECYCLE idle, GuestStatus idle (once per busy period)
response with success: false EVENT_NOTICE, title = the command, text = error, is_error
subagent_lifecycle EVENT_SUBAGENT, actor = payload.id (or index, or subagent), title = status, text = description or agent
subagent_progress EVENT_SUBAGENT, actor as above, title = agent, text = task and the progress summary (≤ 2 KiB)
notice EVENT_NOTICE, title = level, error if level = "error"
auto_retry_start EVENT_NOTICE retry, attempt a/b in N ms: <errorMessage>
auto_compaction_start / _end EVENT_NOTICE compaction
rpc_frame_error EVENT_NOTICE rpc_frame_error (error)
extension_ui_request See below
anything else ignored

After protocol v2 is negotiated, omp splits oversized stdout objects into rpc_chunk frames (chunkId, index, count, byteLength, base64 data). omp::Chunks reassembles strictly: a sequence must start at index 0, keep the same chunkId/count/byteLength, arrive in order with no other frame in between, and decode to exactly byteLength bytes, which must not exceed MAX_REASSEMBLED (64 MiB). Any violation drops the sequence and emits an EVENT_NOTICE rpc_chunk error; a non-chunk frame in the middle interrupts it (“sequence interrupted”). The reassembled frame must be a JSON object and is dispatched like any other frame.

method Guest action
confirm ApprovalRequest{question: title + "\n\n" + message, options: []}
select ApprovalRequest{question: title, options}
input, editor cancelled right away
notify EVENT_NOTICE, title = notifyType
cancel forget the pending request whose UI id is targetId

timeout (ms) becomes timeout_secs (rounded up). Approval ids come from a process-wide counter (NEXT_APPROVAL), so answers to a previous omp process never match a new request.

Claude Code stream-json usage by the guest

Section titled “Claude Code stream-json usage by the guest”

With agent.harness = "claude_code" the guest runs Claude Code headless, one long-lived process, JSON lines both ways. Like omp, the protocol handling is a pure state machine (Claude) and the process plumbing is harness::run_session. The message shapes follow the Agent SDK’s SDKMessage types and the headless-mode docs as published for Claude Code 2.1.319, the version the tools image pins.

Command line (an empty agent.command means claude):

claude -p --input-format stream-json --output-format stream-json --verbose
--permission-mode bypassPermissions --strict-mcp-config
--mcp-config /home/agent/.gopala/claude-mcp.json
--append-system-prompt-file /home/agent/.gopala/system.md
[--tools=a,b] <agent.args> [--continue]

--append-system-prompt-file keeps Claude Code’s own system prompt and adds Gopala’s after it; --strict-mcp-config makes the generated file the only MCP config, so a .mcp.json the agent leaves in its workspace changes nothing. --continue is added once /home/agent/.gopala/claude-started exists, which the guest writes on the first system/init line, so a fresh install never asks to continue a session that doesn’t exist. The environment gets CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 before HostInit.env, so agent.env can override it.

Claude Code prints nothing before its first input, so the process counts as ready once spawned (lifecycle ready, status ready) and queued deliveries go out at once. Each delivery is one line:

{"type":"user","message":{"role":"user","content":"alice [message 7]:\n..."},"parent_tool_use_id":null}

Claude Code has no separate steer or follow-up. While a turn runs, a delivery carries "priority": "later" (read after the turn, journaled as follow_up); an urgent one goes without a priority, which Claude Code reads as soon as the running tool calls finish (journaled as steer). Abort has no command either: while a turn runs, the guest stops the process and starts it again with --continue; while idle, there’s nothing to stop.

Claude Code line Mapping
assistant with text / thinking blocks EVENT_TEXT / EVENT_THINKING, one event per block (split above 16 KiB); a first assistant line without a prompt from the guest (a queued message, a background task) also means busy
assistant with tool_use / server_tool_use blocks EVENT_TOOL_CALL, title = name, text = input as JSON; the id is remembered (up to 4096 open calls) to name the result
assistant with error EVENT_NOTICE error (error)
user with tool_result blocks EVENT_TOOL_RESULT, title = the tool’s name, text = the string or the text blocks joined, is_error
user with a string, from a subagent EVENT_SUBAGENT, title prompt, text = the subagent’s task
result EVENT_LIFECYCLE idle, GuestStatus idle; when is_error or the subtype isn’t success, first an EVENT_NOTICE titled with the subtype and the errors (or result) as text
system / init writes the claude-started marker; EVENT_LOG init with the version and model; an EVENT_NOTICE mcp (error) for every MCP server whose status isn’t connected or pending
system / api_retry EVENT_NOTICE retry, attempt a/b in N ms: <error>
system / compact_boundary EVENT_NOTICE compaction, the trigger and the token count before it
anything else, unparseable lines ignored (unparseable ones logged)

Lines from a subagent carry parent_tool_use_id; their events use that id as the actor, and their text isn’t coalesced. A delivery while idle is journaled as prompt and marks the session busy right away. Approval frames from the host are ignored: with bypassPermissions Claude Code never asks, so approvals only happen through the request_approval chat tool. Not exercised against a real Claude Code process yet: the priority field, --continue after SIGTERM and the timing of init are as documented, not as observed.

Served by the host’s web server (watch), which upgrades GET /watch and hands the websocket to Rooms::viewer; Rooms::accept does the same on a raw stream (tests). The reference client is gopala_rooms::client::watch, used by gopala watch.

With gopala run, a valid token isn’t enough: the holder must still be a member of the conversation and match [monitoring] output (see Who may read what), checked at hello (not allowed to watch this conversation) and again whenever the membership answer expires (access revoked, then the socket closes). Rooms::viewer_authorized asks the host through a gopala_rooms::Authorize callback (web::watch_authorization).

  • ws:// or, with server.tls_cert/tls_key, wss:// (rustls, ring provider, safe default protocol versions, ALPN http/1.1) at /watch on server.bind. Only in gopala run with rooms.enabled; otherwise /watch answers 404. A request without a valid websocket upgrade (Sec-WebSocket-Key, version 13, Connection: upgrade, Upgrade: websocket) gets 426. The socket is created with gopala_rooms::websocket_config() (4 KiB message and frame limit). Every message is one binary websocket message holding one protobuf frame; text messages are an error; websocket pings are answered by the library.
  • A watch link is <watch_url>#<token>, where watch_url is server.public_url with http(s) turned into ws(s) plus /watch (Server::watch_url). The token travels only inside the hello frame, never in the URL that is requested.
  • Token format: gpl1. + base64url-without-padding of a random 12-byte id (16 characters) + . + base64url-without-padding of a random 32-byte secret (43 characters). Anything else is rejected before a database lookup.
  • The server stores SHA-256(secret) and the key-wrapping key derived from it, never the secret. Verification looks up the id, deletes the row if expired, and compares hashes in constant time (subtle). Issuing a token replaces the user’s previous token for that scope.

ClientFrame (viewer → server):

Variant Fields Notes
hello (1) ViewerHello{token, since_seq, live_only} Must be the first frame. since_seq: replay events with seq > since_seq (0 = from the start); live_only: skip the replay.
ping (2) Ping{nonce} Answered with pong carrying the same nonce. Any other frame after hello closes the connection.

ServerFrame (server → viewer):

Variant Fields When
welcome (1) Welcome{scope, title, encryption: ENCRYPTION_NONE | ENCRYPTION_SEALED, latest_seq} First, after a valid hello
key (2) RoomKey{epoch, nonce, wrapped} Sealed mode: right after welcome, then at every rotation
event (3) SessionEvent none mode: each event
sealed (4) SealedEvent{epoch, nonce, ciphertext} Sealed mode: each event
history_done (5) HistoryDone{latest_seq} After the replay (also sent when live_only)
pong (6) Ping{nonce} Reply to ping
error (7) ErrorFrame{message} Before the server closes: expected hello, invalid or expired token, room is full, access revoked, the agent left this conversation, too slow, reconnect with since_seq

Order on a connection: welcome, key (sealed), replayed events for since_seq < seq ≤ latest_seq in pages of 256, history_done, then live events with seq > latest_seq, key rotations and pongs. Events are delivered in seq order without gaps or duplicates: the viewer registers and subscribes to the live channel under the same lock that publish holds when it advances last_seq, so everything up to latest_seq is in the database and everything after arrives live. A viewer that falls more than 1024 live events behind is disconnected with too slow, reconnect with since_seq and can resume with since_seq = the last seq it saw (in sealed mode the seq is inside the decrypted event).

Implemented in the watch service with RustCrypto XChaCha20Poly1305, hkdf and sha2.

Item Definition
Key-wrapping key (derive_kwk) kwk = HKDF-SHA256(salt = none, IKM = token secret (32 bytes), info = "gopala-kwk|" ‖ scope, L = 32)
Room key (RoomKey) 32 random bytes; epoch: u32 starts at 1 per room (per host process) and increments on rotation (wrapping, never 0)
RoomKey.wrapped (wrap_key) XChaCha20-Poly1305(key = kwk, nonce = 24 random bytes, AAD = "gopala-room-key|" ‖ scope ‖ "|" ‖ epoch as u32 big-endian, plaintext = room key); RoomKey.nonce = the nonce
SealedEvent.ciphertext (seal_event) XChaCha20-Poly1305(key = room key of this epoch, nonce = 24 random bytes, AAD = "gopala-event|" ‖ scope, plaintext = encoded SessionEvent with seq set); SealedEvent.epoch names the key
Nonces 24 bytes, random per message (rand::random); the client rejects other lengths

Binding the scope into the HKDF info and both AADs means a key or event from one room doesn’t open in another; binding the epoch into the key AAD prevents relabeling a wrapped key. Each live event is sealed once and the same bytes are fanned out to every viewer; replayed events are sealed with the key current when the viewer joined.

Rotation happens every rooms.key_rotation_secs (rotate_periodically), when a viewer’s access is revoked (Rooms::revoke, on MembersChanged.left), and when the bot leaves (Rooms::revoke_scope). Ordering guarantee: Room::rotate runs under the room’s state lock and queues the new wrapped key on every viewer’s control channel before it replaces the key that publish (holding the same lock) seals with; the viewer loop polls its control channel before live events (biased select). So a viewer always receives a key frame before any event sealed under that epoch. Because key frames overtake queued live events, a burst of rotations can deliver several new keys before events sealed under an older epoch; the client therefore keeps the 16 newest keys (KEEP_KEYS in the viewer client).

In none mode events are sent as plain SessionEvents and confidentiality relies on TLS.

Limit Value
Websocket handshake (when Rooms::accept does it) + hello 10 s (HELLO_TIMEOUT); behind /watch the clock starts after the HTTP upgrade
Client frame / message size 4096 bytes (MAX_CLIENT_FRAME)
Viewers per room rooms.max_viewers_per_room (default 16)
Live buffer per room 1024 events (LIVE_CAPACITY)
Replay page 256 events (REPLAY_PAGE)
Title 256 bytes (MAX_TITLE)
Client-side accepted frame 2 MiB (2 * MAX_FRAME, client::watch)
Token lifetime rooms.token_ttl_secs (default 7 days)

/monitor lets any app ask about a conversation’s agent and follow it live: its status, its VM’s metrics, usage and budget, and the journal (every task, tool call, subagent, notice) as SessionEvents. It is request/response over protobuf like Firefly’s own protocol, with pushed events for subscriptions. Implemented by the host’s monitor; the messages are in the “Monitor websocket” section of the protobuf schema. The same answers are served over HTTP under /api/monitor/.

GET /monitor on server.bind, upgraded to ws / wss like /watch (426 without a valid upgrade). Each message is one binary websocket message holding one MonitorClientMessage or MonitorServerMessage; text messages or undecodable bytes close the connection. Client messages and frames are at most 16 KiB. In gopala web every request answers 409.

message MonitorClientMessage { oneof message { MonitorRequest request = 1; bytes ping = 2; } }
message MonitorServerMessage {
oneof message { MonitorResponse response = 1; MonitorEvent event = 2; MonitorMetrics metrics = 3; bytes pong = 4; }
}
message MonitorRequest {
uint32 id = 1; // the client's; the response echoes it
oneof payload {
Authenticate authenticate = 2; ListConversations list = 3; ConversationRef status = 4;
ConversationRef metrics = 5; UsageRequest usage = 6; EventsRequest events = 7;
SubscribeRequest subscribe = 8; ConversationRef unsubscribe = 9;
}
}
message MonitorResponse {
uint32 id = 1;
MonitorError error = 2; // code 400, 401, 403, 404, 409, 429 or 500, and a message; body empty
oneof body {
Identity identity = 3; ConversationList conversations = 4; ConversationStatus status = 5;
MonitorMetrics metrics = 6; UsageHistory usage = 7; EventPage events = 8; bool ok = 9;
}
}

ping is answered with pong carrying the same bytes.

A connection is anonymous until it authenticates, in one of two ways:

  • On the upgrade: Authorization: Bearer <token> (any kind below). A wrong token answers the upgrade with 401 (or 429 while locked out) and no websocket. The dashboard’s session cookie also authenticates the upgrade as the operator, but only when the request has no Origin or its Origin host equals Host, so another site’s page can’t open a socket with it.
  • First message: a MonitorRequest with authenticate, within 10 s of the upgrade. No message in time: the server closes. Any other request first, or a failed authenticate: an error response (401) and the connection closes.
message Authenticate { oneof credential { string lupyd_token = 1; string watch_token = 2; string admin_token = 3; } }
message Identity { string platform = 1; string username = 2; string only_conversation = 3; }
Credential Who it is Checks
admin_token the operator (Identity.platform = "operator"): everything constant-time compare with the admin token
watch_token the person the bot sent that /watch link to, on that conversation’s platform; only_conversation is the one conversation it reaches Rooms::token_holder (format, hash, expiry); checked again before each request and subscription check, since leaving revokes it
lupyd_token a person signed in to Lupyd (platform lupyd), any of their Lupyd conversations only with monitoring.lupyd_tokens (default on): an Auth0 JWT, see Lupyd tokens

A Bearer token is sorted by shape (authenticate_bearer): the admin token if it matches, gpl1.… as a watch token, three dot-separated parts as a Lupyd token, anything else as a (failed) admin token. Failed admin and watch tokens count towards the address’s sign-in lockout (more than 10 a minute → 429 for 60 s, shared with the dashboard’s sign-in); bad Lupyd tokens don’t.

authenticate again on an open connection replaces the identity and drops every subscription. When a Lupyd token’s exp passes (30 s leeway) or a watch token is revoked, the next request answers 401 credentials expired; authenticate again and the connection stays open, unauthenticated, for a new authenticate.

The Lupyd app’s own Auth0 access token, checked by auth::LupydTokens::verify:

  • at most 8 KiB; RS256 only; the header must name a kid;
  • signed by one of the RSA signing keys in {platforms.lupyd.auth0_domain}/.well-known/jwks.json (at most 16 kept; fetched without redirects, ≤ 64 KiB, 10 s timeout). Keys are fetched again after an hour, and for an unknown kid at most once a minute; a failed fetch keeps the old keys;
  • iss = {auth0_domain}/, aud = platforms.lupyd.auth0_audience, exp, iss and aud required, nbf checked when present, 30 s leeway either way;
  • the username is the uname claim (1–128 bytes), the same claim Firefly reads.

Every request about a conversation is checked (Control::grant):

  1. The operator may read everything.
  2. A person must be on the conversation’s platform, and a watch token’s holder only reaches its own conversation.
  3. They must be a member now: in a group, the provider answers Platform::member (cached, below); in a direct chat, the peer is the only member and counts as its owner.
  4. [monitoring] decides what that member may read: output (the journal and live events) and metrics (metrics, usage, budget). The conversation’s [[monitoring.conversations]] entry replaces the defaults (output = ["members"], metrics = ["approvers"]). Any matching rule grants: members, approvers (as [approvals] approvers decides), owner, role:<name> and permission:<name> (from the provider’s answer, case-insensitive), user:<username>. An empty list leaves it to the operator.
Request Needs
list nothing; returns every conversation where the caller may read output or metrics, each with can_read_output and can_read_metrics
status output or metrics
metrics, usage metrics
events output
subscribe output for output = true, metrics for metrics = true

Someone who isn’t a member gets 404 no such conversation, the same answer as for a key that doesn’t exist, so the API never confirms that a conversation exists. A member the rules don’t allow gets 403.

Membership cache (auth::Membership): answers are kept per (conversation key, username as the platform spells it, case-sensitive) for monitoring.membership_cache_secs (default 60, at most 3600), at most 4096 entries. A provider error counts as “not a member” and isn’t cached. Event::MembersChanged drops the answers for everyone who joined or left, Event::Updated and the bot leaving drop the whole conversation, and every drop makes live subscriptions check again.

Request Body Notes
list (ListConversations {}) ConversationList{conversations: [ConversationSummary{key, platform, direct, title, status, can_read_output, can_read_metrics}]}
status (ConversationRef{key}) ConversationStatus{conversation, status_since_ms, error, last_activity_ms, latest_seq} status as on the dashboard (asleep, working, …)
metrics (ConversationRef{key}) MonitorMetrics{key, at_ms, allocation, cpu_percent, memory_rss_mib, disk_used_mib, net_rx_bytes, net_tx_bytes, running, today, total, budget_used_today, budget_daily, budget_used_month, budget_monthly, budget_blocked} measured on the host; zeros and running = false while asleep; today/total are UsageTotals, which include git_requests (14), git_bytes_up (15) and git_bytes_down (16) for the git proxies
usage (UsageRequest{key, days}) UsageHistory{key, days: [UsageDay{day, usage}]}, newest first days 1–90, 0 = 30
events (EventsRequest{key, after, limit, latest}) EventPage{key, events: [SessionEvent], latest_seq}, oldest first events with seq > after; after = 0 with latest = the newest limit; limit 0 = 200, at most 500
subscribe (SubscribeRequest{key, after, output, metrics}) ok see below
unsubscribe (ConversationRef{key}) ok 404 not subscribed if there was none

A subscription follows one conversation until it’s cancelled, replaced or loses access; pushed messages carry the key:

  • With output: the journal after after is replayed first (pages of 200; after = 0 replays nothing), then live events as MonitorEvent{key, event}. If the connection falls behind the live broadcast, the missed events are read back from the journal.
  • With metrics: one MonitorMetrics right away, then one every 5 s while the VM runs.
  • At least one of output and metrics (400 otherwise). At most 16 subscriptions per connection (429 beyond); subscribing to the same conversation again replaces the earlier one.
  • Access is checked again when the membership answer expires (at most once a second) and whenever the membership cache drops answers. When the caller loses access, or the conversation goes away, the subscription ends with a MonitorResponse that has the subscribe request’s id and the error: 401 (credentials expired), 403 or 404.
Limit Value
Client message and frame 16 KiB
Authenticate within 10 s, unless the upgrade did
Requests and pings 20 a second, bursts of 40 per connection; beyond: 429 slow down
Subscriptions 16 per connection
Messages queued for the client 256; subscriptions wait when it’s full

One proxy per [[llm]] entry i.

sequenceDiagram
    participant O as omp (agent user)
    participant F as guest forwarder
    participant P as host llm_proxy
    participant U as upstream
    O->>F: HTTP to 127.0.0.1:guest_port, placeholder key
    F->>P: same bytes over vsock 2:(1100+i)
    P->>P: origin-form check, read body ≤ 32 MiB, rewrite headers
    P->>U: HTTPS upstream + path?query, auth_header: auth_prefix + secret
    U-->>P: status, headers, body (SSE)
    P-->>F: status, filtered headers, body streamed as it arrives
    F-->>O: same bytes
  • Guest side (the guest agent’s forward): a TCP listener on 127.0.0.1:<guest_port>; each connection is piped byte for byte to vsock 2:<1100+i> (copy_bidirectional). The harness finds it through guest_env, e.g. ANTHROPIC_BASE_URL = "http://127.0.0.1:8701" with a placeholder ANTHROPIC_API_KEY.
  • Host side: hyper HTTP/1 server per connection (keep-alive allowed). For each request:
    • Only origin-form targets (/path?query) are accepted; absolute-form (http://host/…) and authority-form (CONNECT) get 400 origin-form requests only.
    • The body is read fully, at most 32 MiB (MAX_REQUEST_BODY); larger or broken bodies get 413 request body too large or broken.
    • URL = upstream without its trailing / + the request’s path and query; method preserved.
    • Request headers dropped: hop-by-hop and framing (host, connection, keep-alive, proxy-connection, te, trailer, transfer-encoding, upgrade, content-length), every credential header a guest could use (authorization, proxy-authorization, proxy-authenticate, x-api-key, api-key, x-goog-api-key), and the configured auth_header. Then auth_header: <auth_prefix><secret> is set (marked sensitive). The header name is lowercased; auth_prefix is empty by default (use "Bearer " for authorization).
    • The request asks the upstream for accept-encoding: identity whatever the guest asked for, so the provider’s usage report stays readable.
    • Response: upstream status; headers minus the same dropped set; body streamed frame by frame, so server-sent events pass through as they arrive.
    • Errors: 502 upstream unreachable (connect/send failure, logged) or bad upstream response, as {"error":{"type":"gopala_proxy","message":"…"}} with content-type: application/json.
    • The upstream client is reqwest with rustls and a 30 s connect timeout; there is no overall request timeout (long streams are expected). Redirects are never followed (Policy::none()): a 3xx goes back to the guest unchanged, so the key never reaches another host.
  • Metering: each request produces one LlmCall (upstream, model, status, duration, bytes up and down, input/output/cache-read/cache-write tokens, error), reported when the response body ends, fails, or the guest drops it, and also for the proxy’s own refusals. The model is the request body’s model, else a Gemini-style /models/{model}:… path, else the one the response names. Tokens come from the provider’s usage report, read by a parser the response is teed through as it streams (Content-Type: text/event-stream → SSE data: documents, otherwise one JSON document or a JSON array of them), holding at most MAX_PARSED (8 MiB) at a time: Anthropic usage (input_tokens, output_tokens, cache_creation_input_tokens, cache_read_input_tokens), OpenAI Chat Completions (prompt_tokens, completion_tokens, prompt_tokens_details.cached_tokens), OpenAI Responses (input_tokens, output_tokens, input_tokens_details.cached_tokens) and Gemini usageMetadata (promptTokenCount, candidatesTokenCount, thoughtsTokenCount, cachedContentTokenCount). Cached tokens that providers count inside the prompt are subtracted, so input + cache_read + cache_write is the whole prompt; output includes reasoning tokens. A response with a content-encoding other than identity is forwarded untouched and counted with zero tokens. The session books the call into the usage ledger and journals it as EVENT_LLM.
  • Secrets are read once from secret_env at startup (main::secrets) and removed from the environment; a missing variable is a startup error.

Budget refusals. Before forwarding, the proxy asks the conversation’s budget (budget::check through budget::gate). A refused call never leaves the host: the guest gets 429 Too Many Requests with the Anthropic error shape, which OpenAI-style clients read too (.error.message):

{"type": "error", "error": {"type": "budget_exceeded", "message": "This conversation used its daily budget (2000000 of 2000000 tokens). It resets at 00:00 UTC."}}

The refusal is metered as a call with status 429 and no tokens. The first refusal after calls were allowed is journaled as an EVENT_NOTICE titled budget; the next one is journaled only after a call goes through again. People hear of it as a host event (HostConversation::host_event, “⛔ I’ve stopped: ” at alerts.host_events severity): once per episode when the conversation’s own budget stopped it, once per UTC day when the host budget did (budget::Announced). The check happens before the call, so the call that crosses a limit finishes and is counted in full.

Only with vm.network = "proxy". The VM has no network device; HostInit.egress_proxy makes the guest forward 127.0.0.1:3128 to vsock port 1090 and set http_proxy, https_proxy (and the uppercase forms) to http://127.0.0.1:3128 with no_proxy=127.0.0.1,localhost for the harness, provisioning, login shells (/etc/profile.d/gopala-proxy.sh) and apt (/etc/apt/apt.conf.d/90gopala-proxy).

One guest connection is one proxy session (hyper HTTP/1, keep-alive):

  • CONNECT host:port opens a tunnel: 200 and then raw bytes both ways, so TLS stays end to end between the program and the server.
  • An absolute-form GET http://host[:port]/path (any method) is forwarded over a fresh connection with hop-by-hop headers removed and Host set; the response comes back the same way, redirects included.
  • Anything else: 400 (only absolute http:// URLs, or CONNECT for https).

Every target is vetted first (Egress::vet):

  1. host:port must match vm.allowed_hosts (Allowlist): host matches exactly, *.domain any subdomain (not the domain itself), with an optional :port, else ports 80 and 443. Else 403 (<host:port> is not in vm.allowed_hosts).
  2. The name is resolved on the host (system resolver, 10 s). No answer: 502.
  3. Every resolved address must be public (egress::is_public): not private, loopback, link-local, CGNAT, multicast, broadcast, unspecified, reserved, benchmarking or documentation space, nor an IPv6 ULA, link-local, site-local, Teredo, discard-only or documentation address, nor an IPv4-mapped, NAT64 (64:ff9b::/96) or 6to4 address embedding one of those. One bad address refuses the whole name: 403.
  4. The host connects to one of those vetted addresses (10 s each), never resolving the name a second time. None answers: 502.

Refusals are plain text for the guest. Each allowed or refused target is journaled as an EVENT_NOTICE titled network (allowed host:port, refused host:port (reason)), at most 30 a minute with bursts of 10, the same refused target at most once a minute, and what was held back counted into the next notice. Bytes in both directions are booked to the conversation (egress_bytes_up, egress_bytes_down).

Limits per VM (egress::Limits): 64 connections at once (more are refused and journaled), request heads ≤ 16 KiB and 64 headers, 30 s for a request head, connections and tunnels closed after 300 s without a byte either way.

An agent.mcp entry with a url is reached through the host in every network mode (egress::McpUpstream, egress::serve_mcp). The guest’s forwarder on 127.0.0.1:3200+i carries each connection to vsock 1200+i; the host:

  • accepts origin-form requests only and sends them to the configured URL’s origin with the same path and query, so the guest can’t pick another host;
  • drops hop-by-hop headers and every header a guest could put a credential in (authorization, cookie, x-api-key, api-key, proxy-authorization, …), then adds <auth_header>: <auth_prefix><secret> from secret_env when there is one (default authorization: Bearer …);
  • limits request bodies to 16 MiB, never follows redirects, streams responses (server-sent events included), and books bytes as egress;
  • allows 64 connections at once and 30 s for a request head.

Secrets are read at startup. A server added by a hot reload works right away if it needs no secret; one that needs a secret waits for a restart, and the guest’s connections to it fail until then (egress::live_mcp).

One proxy per [[git]] entry i, in every vm.network mode. The guest’s forwarder on 127.0.0.1:3300+i carries each connection to vsock 1300+i; the host forwards over HTTPS to the account’s code host (url, default https://github.com or https://gitlab.com) and adds the token, which never enters the VM.

Guest setup. From HostInit.git the guest writes /etc/gopala-git.conf, included from /etc/gitconfig: url.<local>.insteadOf rewrites https://<host>[/path]/, git@<host>: and ssh://git@<host>/ to http://127.0.0.1:<3300+i>/, with an empty credential helper for those URLs. When two accounts share a host only the first gets the rewrite; the second is used through its own http://127.0.0.1:<port>/ URL. The API base is exported as GOPALA_GIT_<NAME>_API (in the harness environment and /etc/profile.d/gopala-git.sh).

Endpoints. Only these paths are forwarded; anything else answers 404 with a hint:

Kind Paths Methods
Ref advertisement <repo>[.git]/info/refs?service=git-upload-pack or git-receive-pack GET
Fetch <repo>[.git]/git-upload-pack POST
Push <repo>[.git]/git-receive-pack POST
REST API GitHub /api/… (to https://api.github.com/…, or <url>/api/v3/… on Enterprise Server); GitLab /api/v4/…; Gitea /api/v1/… (git_api_path) as sent, subject to the checks below

Dumb HTTP (info/refs without service) and other services are 403; the wrong method on a git path is 405. GraphQL, LFS and the gh/glab CLIs are not supported; agents use the REST API.

Authentication the host adds.

Provider Git smart HTTP REST API
GitHub Basic x-access-token:<token> Authorization: Bearer <token>
GitLab Basic oauth2:<token> Authorization: Bearer <token>
Gitea Basic gopala:<token> Authorization: token <token>

Headers. Requests keep only accept, accept-encoding, accept-language, content-type, content-encoding, git-protocol, user-agent (default gopala-git-proxy), if-none-match, if-modified-since and x-github-api-version; the guest’s Authorization and Cookie never reach the code host. Responses lose set-cookie, www-authenticate and any header containing the token. Redirects are not followed; the token is redacted from logs.

Refusals. Every refusal is a plain-text body starting Gopala host:; 403s are also journaled as git notices (rate-limited).

Status When
400 absolute-form request (origin-form requests only); empty, . or .. segments (also %2e); not a repository path; a _method override; a signed push (push-cert); a malformed or incomplete push command list
403 repository not in repos; a numeric repository id with repos set (GitLab /projects/<id>: use the URL-encoded path; GitHub /repositories/<id>); a write naming no repository with repos set (only GET/HEAD); GraphQL; merging a pull or merge request (GitHub PUT pulls/{n}/merge, GitLab PUT merge_requests/{iid}/merge, Gitea POST pulls/{n}/merge) or GitLab DELETE repository/merged_branches; deleting, force-updating or renaming a protected branch; a commit endpoint (GitHub contents, merges, merge-upstream; GitLab files, commits, cherry_pick, revert; Gitea contents, diffpatch, merge-upstream) whose branch is protected or not named in the body or query; dumb HTTP or another git service
404 not a git smart HTTP or API path (dumb HTTP files, LFS, other API prefixes)
405 wrong method for a git path
408 a push’s command list didn’t arrive within 120 s
413 API body over 10 MiB, push command list over 64 KiB
415 a compressed push body (Content-Encoding other than identity)
502 the code host is unreachable, timed out, or sent a bad or truncated answer
503 the host is shutting down

A push that creates, updates or deletes refs/heads/<protected> is refused whole. When the client asked for report-status (git always does), the answer is a receive-pack result in which git prints ! [remote rejected] main -> main (protected by the Gopala host) and remote: Gopala: …; the push’s other refs are rejected as not pushed: the same push updates a protected branch. Without report-status it is a 403.

Limits. Pack uploads ≤ 2 GiB, 64 connections per account, 16 requests in flight per VM, 30 s for headers and to connect, 300 s read idle, 120 s for a whole API call.

Journal and usage. Successful pushes, branch deletions, tags and opened pull or merge requests become git notices (pushed branch X to o/r, opened pull request #7 on o/r: <url>, …). Requests and bytes are booked as git_requests, git_bytes_up and git_bytes_down.

The one web server (axum on hyper HTTP/1, [server]) serves the static site from server.site_dir (front page at /, /templates/, /dashboard/, /configure/, /architecture/, /docs/), the viewer websocket at /watch, the monitor websocket at /monitor, and the API under /api/. It runs in gopala run (Mode::Run) and in gopala web (Mode::Web: site and configuration API only; the control and monitor endpoints answer 409 {"error": "gopala web runs no agents; start gopala run"}). Every endpoint except /api/status, POST /api/session and the static site needs a credential.

Every response carries X-Content-Type-Options: nosniff, Referrer-Policy: no-referrer, X-Frame-Options: DENY and Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; font-src 'self'; connect-src 'self'; frame-ancestors 'none'; base-uri 'self'; form-action 'self' (CSP). Request bodies are limited to 256 KiB (BODY_LIMIT). A client must send its request headers within 15 s (HEADER_TIMEOUT, also bounding idle keep-alive connections) and finish a TLS handshake within 10 s (TLS_TIMEOUT). Errors are {"error": "…"}; API responses that carry state or configuration are Cache-Control: no-store. Unknown paths fall through to the site (with its 404.html if present); without a site directory / explains that only the API is served.

The admin token comes from the environment variable named by server.admin_token_env (default GOPALA_ADMIN_TOKEN), taken out of the environment at startup. Unset or empty: admin endpoints answer 403 {"error": "admin API disabled"}.

Endpoint Request Response
GET /api/status (public) — {"mode": "run" | "web", "admin": bool, "signed_in": bool}, plus "version" once signed in; admin: a token is configured; signed_in: this request carries a valid session cookie or the admin Bearer token
POST /api/session {"token": "…"} 204 with Set-Cookie: gopala_session=<32 random bytes, base64url>; HttpOnly; SameSite=Strict; Path=/; Max-Age=43200 (; Secure with server.tls_cert or an https:// public_url); 400 malformed body, 401 wrong token, 403 admin API disabled, 429 locked out
DELETE /api/session — 204, the session forgotten and the cookie cleared

Every admin endpoint accepts either Authorization: Bearer <token> or the session cookie (web::authorize). Tokens are compared in constant time (subtle::ConstantTimeEq). Sessions live 12 hours (SESSION_TTL) in the server’s memory, at most 256 at once (the oldest is dropped). Cookie-authenticated requests other than GET/HEAD must send X-Gopala: 1, which a cross-site form can’t, else 403 {"error": "missing X-Gopala header"}. A missing or wrong credential is 401 {"error": "unauthorized"} with WWW-Authenticate: Bearer.

Lockout (web::Guard): every wrong token, from POST /api/session or a Bearer header, counts against the client’s IP address; more than 10 failures within a minute (FAILURES_PER_MINUTE) lock that address out for 60 s (LOCKOUT). While locked out, sign-in and every admin request answer 429 {"error": "too many failed sign-ins; try again later"} with Retry-After: <seconds>, even with the right token. The address is the TCP peer, so behind a reverse proxy all clients share the proxy’s address.

Admin only; 409 in gopala web; 404 {"error": "no such conversation"} for an unknown key. Implemented by the host’s control API.

Endpoint Request Response
GET /api/state — The live snapshot (Control::state), see below
GET /api/stream — text/event-stream: event state (data = the /api/state JSON) on connect and then whenever it changed, checked every 2 s; event activity (data = {"key", "event": Event}) for every journaled event; a keep-alive comment every 15 s. Cookie auth works (EventSource sends cookies same-origin). Activity a slow client misses is skipped.
GET /api/conversations/{key}/events?after=<seq>&limit=<n> — {"journal": bool, "events": [Event]}, oldest first: events after after, or the newest limit without it; limit 1–500, default 200. journal: false (and no events) when [rooms] is off.
GET /api/usage?days=<n> — {"days": [{"day": "YYYY-MM-DD", "scopes": {"<key>": Usage}}]}, newest first; days 1–90, default 30
POST /api/conversations/{key}/abort — 202: abort the agent’s current turn and drop queued messages
POST /api/conversations/{key}/sleep — 202: stop the VM now (the next message wakes it)
POST /api/conversations/{key}/restart — 202: restart the VM, or boot it if asleep; a pending allocation applies
POST /api/conversations/{key}/leave — 202: the bot leaves the conversation (Platform::leave) and its session ends; watch tokens are revoked
PUT /api/conversations/{key}/allocation {"vcpus"?, "memory_mib"?, "disk_gib"?} (totals) 200 {"granted", "after_restart": Allocation, "message"}. An operator override (VmControl::operator_set): the grants policy and approvals don’t apply; the per-VM maximums, the memory budget, the host disk reserve and “disks only grow” do. Applies at the next boot. 400 on a malformed body.
PUT /api/conversations/{key}/budget {"daily_tokens"?, "monthly_tokens"?, "priority"?: "low" | "normal" | "high"} 200 {"saved": true}. Upserts the conversation’s [[budget.conversations]] entry (fields left out keep its current effective value), edits gopala.toml with toml_edit so comments and formatting survive, validates, writes it atomically (.bak kept) and applies it at once: the next model call and boot see it. 400 malformed body, 422 if the edited file doesn’t validate.
PUT /api/conversations/{key}/behavior {"respond_to": "all" | "mentions" | "dms"} 200 {"saved": true}. Listening mode: upserts the conversation’s [[behavior.conversations]] entry the same way (control::set_respond_to) and applies it to the next message. 400 malformed body, 404 unknown conversation, 422 invalid result.

Every change to gopala.toml (PUT /api/config, budgets, listening mode, and /listen in the chat) goes through one writer, control::ConfigFile, which holds a lock while it reads, edits, validates and atomically writes the file. Each action except the budget edit is journaled as an EVENT_NOTICE titled operator (The operator …).

/api/state (in run mode):

{
"version": "0.1.0", "mode": "run", "started_at_ms": 0,
"host": {"memory_budget_mib": 0, "memory_used_mib": 0, "slots_total": 8, "slots_used": 1,
"disk_free_bytes": 0, "data_dir": "/var/lib/gopala"},
"limits": {"base": {"vcpus": 2, "memory_mib": 2048, "disk_gib": 10},
"maximum": {"vcpus": 4, "memory_mib": 8192, "disk_gib": 50}, "grants": "auto"},
"budget": {"daily_tokens": 0, "monthly_tokens": 50000000, "used_today": 812000, "used_month": 9100000,
"low_stops_at_percent": 70, "normal_stops_at_percent": 90},
"platforms": [{"name": "lupyd", "bot": "helper", "conversations": 3}],
"conversations": [{
"key": "lupyd-g-42", "platform": "lupyd", "kind": "group", "id": "42", "title": "Acme",
"status": "working", "status_since_ms": 0, "error": null,
"allocation": {"vcpus": 2, "memory_mib": 2048, "disk_gib": 10}, "after_restart": null,
"measured": {"cpu_percent": 12.5, "memory_rss_mib": 900, "disk_used_mib": 3100,
"net_rx_bytes": 0, "net_tx_bytes": 0, "sampled_at_ms": 0},
"reported": {"memory_used_percent": 40, "memory_available_mib": 1200, "disk_used_percent": 31,
"disk_free_mib": 6800, "load": 0.5, "age_seconds": 3},
"budget": {"priority": "normal", "daily_tokens": 2000000, "monthly_tokens": 0,
"used_today": 310000, "used_month": 2400000, "blocked": null},
"respond_to": "mentions",
"usage": {"llm_requests": 0, "input_tokens": 0, "output_tokens": 0, "cache_read_tokens": 0,
"cache_write_tokens": 0, "llm_bytes_up": 0, "llm_bytes_down": 0, "llm_errors": 0,
"tool_calls": 0, "messages_sent": 0, "boots": 0, "cpu_seconds": 0, "vm_seconds": 0,
"egress_bytes_up": 0, "egress_bytes_down": 0, "git_requests": 0,
"git_bytes_up": 0, "git_bytes_down": 0},
"usage_today": {"llm_requests": 0},
"last_activity_ms": 0
}]
}
  • status: asleep, waiting for a free VM slot, waiting for host memory, booting, working, idle, or failed (with error); status_since_ms is when it last changed.
  • allocation: what the VM runs with, else what it will boot with; after_restart: a pending allocation while it runs.
  • measured (host, null while asleep): firecracker’s CPU percent since the previous 5 s sample, its resident memory, the disk image’s allocated bytes, tap bytes received and sent.
  • reported (guest, unverified, null while asleep): the last ResourceUsage the guest sent, as percentages with its age.
  • usage / usage_today (usage::Usage, host ledger): all-time and today’s (UTC) totals, same fields (usage_today is abbreviated above). egress_bytes_* count the egress proxy and the HTTP MCP proxies; git_requests and git_bytes_* count the [[git]] proxies.
  • budget (top level): the host’s budget and the tokens all conversations used today and this month (UTC); budget (per conversation): its effective limits and priority (its [[budget.conversations]] entry, else [budget.default]), its usage, and blocked: null, or the plain reason model calls are refused. Tokens are input + output + cache-write; 0 means no limit.
  • respond_to: the conversation’s effective respond_to (its [[behavior.conversations]] entry, else behavior.respond_to); all means listening mode.

An Event (control::event_json) is {"seq", "at_ms", "kind", "actor", "title", "text", "is_error"}, where kind is input, text, thinking, tool_call, tool_result, lifecycle, subagent, notice, log, approval, outbound, llm or unknown (the EventKinds above). Live activity texts are cut to 16 KiB.

Endpoint Auth Request Response
GET /api/config/schema admin — JSON Schema of gopala_config::Config (schemars; doc comments, defaults, enum variants)
GET /api/config/defaults admin — every key at its default, as TOML (text/plain)
POST /api/config/validate admin TOML body 200 {"ok": true} or 422 {"ok": false, "error"} with the same message check-config prints (Config::parse)
POST /api/config/normalize admin TOML body the parsed config re-serialized as complete TOML, or 422 {"error"}
GET /api/config admin — the current gopala.toml text; 404 if there is none
PUT /api/config admin TOML body validated first (422 on error); then written atomically (a 0600 temporary file in the same directory, fsync, the previous file copied to <name>.bak (0600), rename) → {"saved": true, "applied": [...], "restart_required": [...]}

In run mode PUT /api/config also hot-applies what is safe (Control::apply): applied lists behavior, approvals, agent, budget, alerts, monitoring, collaboration and vm limits when they changed; restart_required lists the other top-level sections that differ from the running configuration (for example server, platforms, rooms, llm, image, memory, or vm for settings outside the hot set, such as network and allowed_hosts). If the merged result doesn’t validate, nothing is applied and every differing section is listed. In web mode both lists are empty: everything takes effect at the next gopala run. See architecture.md.

The four helpers (schema, defaults, validate, normalize) need the operator like the rest: 401 without a credential, 403 with no admin token configured, and the cookie’s POSTs need X-Gopala: 1. Signed out, the /configure/ and /templates/ pages use the schema and defaults bundled with the site and can’t check a file against the server.

The HTTP mirror of the monitor websocket’s one-shot requests, with the same identities and rules (monitor::http_list, http_conversation):

Endpoint Query Body
GET /api/monitor/conversations ConversationList
GET /api/monitor/conversations/{key}/status ConversationStatus
GET /api/monitor/conversations/{key}/metrics MonitorMetrics
GET /api/monitor/conversations/{key}/usage days (1–90, default 30) UsageHistory
GET /api/monitor/conversations/{key}/events after, limit (≤ 500, default 200), latest (default: true when after is absent) EventPage

Credentials: Authorization: Bearer <admin token | gpl1 watch token | Lupyd JWT>, sorted by shape as on the websocket, or the dashboard’s session cookie (as the operator). JSON bodies mirror the protobuf messages field by field; events are shaped as on the control API (control::event_json, with kind as a name such as text or tool_call). Errors are {"error": "…"} with the error’s code as the HTTP status (401 adds WWW-Authenticate: Bearer). With Accept: application/x-protobuf the body is an encoded MonitorResponse with id = 0. Responses are Cache-Control: no-store and Vary: accept. Subscriptions need the websocket.