umn/docs/protocol.md

Ultra Mesh Protocol v2

This document describes the protocol implemented by umnd. Multi-byte integers use network byte order unless the binary property-list encoding defines their representation.

Framing and identities

Every peer message is a four-byte unsigned body length followed by a binary property-list WireEnvelope. Bodies are limited to 1,200,000 bytes. An envelope carries protocol version 2 and one WireMessage. Unknown versions terminate the peer session, so v1 peers fail closed.

A node record contains:

• A 32-byte Ed25519 signing public key.
• A 32-byte X25519 agreement public key.
• A 16-byte mesh address equal to 0xfd followed by the first 15 bytes of SHA-256 over the signing public key.

The address is rendered using normal IPv6 text notation. A receiver always recomputes it from the public key before trusting the record. Private keys are created once and stored with mode 0600.

Peer and routing messages

The wire message variants are:

• hello: announces a node record when a transport session becomes ready.
• linkState: carries an origin record, monotonically increasing sequence, sorted neighbor addresses, and the origin's Ed25519 signature over those fields.
• packet: carries a UUID, source and destination addresses, hop limit, and an end-to-end sealed payload.
• keepalive: detects failed transport sessions.

Invalid identities, signatures, stale link-state sequence numbers, oversized neighbor lists, duplicate packet UUIDs, and expired hop limits are discarded.

Each node floods newly accepted link-state records. Records refresh every 10 seconds, expire after 30 seconds, and neighbors are considered lost when their transport connection fails. Routes use shortest-hop Dijkstra calculation with deterministic address ordering for equal-cost paths. Packets start with a hop limit of 16. The duplicate cache retains the most recent 4,096 packet UUIDs.

End-to-end payloads

The sender serializes an InnerFrame, signs its bytes with Ed25519, and encrypts the signed object to the destination record:

1. Generate an ephemeral X25519 key.
2. Perform X25519 agreement with the destination's static agreement key.
3. Derive a 32-byte key using HKDF-SHA256, salt umn-e2e-v1, and the destination address as shared information.
4. Seal with ChaCha20-Poly1305.

The routed payload contains only the ephemeral public key and the combined authenticated ciphertext. The receiver decrypts it, verifies the signature, validates the embedded source record, and confirms that its derived address matches the outer source address. Relays cannot decrypt the inner frame. Neighbor discovery and topology metadata are authenticated but intentionally public in v2.

Inner frame kinds are ping request/reply, text, stream open/data/ack/close/reset, ipv6Packet, and error. Logical destination ports are inside the sealed payload. An ipv6Packet contains the complete IPv6 packet and a signed UUID retained in a bounded destination replay cache. Routed packets mark native bulk traffic so neighbor queues can bound and deprioritize it behind hello, link-state, and keepalive messages.

Native packets must use the authenticated outer source and destination as their IPv6 source and destination. Endpoints accept TCP, UDP, ICMPv6 echo and related errors, plus bounded standard extension-header chains. They reject fragments, multicast, malformed lengths, non-mesh addresses, unsafe routing headers, and unsupported next-header values. Relays route ciphertext and do not inspect the IP header.

Streams and failure handling

A stream uses a random 64-bit identifier. streamOpen occupies sequence zero; data starts at sequence one. Receivers acknowledge the greatest contiguous sequence delivered. Duplicate frames are acknowledged without being delivered again.

Unacknowledged frames are retransmitted through the route table's current next hop. Retry delay begins at 500 milliseconds and backs off to five seconds. No progress for 60 seconds resets the stream. This makes streams independent of any one neighbor TCP connection and allows an access-point path to be replaced by a peer-to-peer or multi-hop path.

Each endpoint enforces these resource bounds:

• 1 MiB aggregate data per stream.
• 32 simultaneous streams per node.
• Eight intended concurrent streams per source; broader abuse controls remain future work.
• 32 KiB bridge read chunks and a logical 64 KiB flow-control target.

New operations are live-route-only and are not written to a durable delivery queue.

Firewall and local IPC

Ping terminates in the daemon and is always available. Firewall rules explicitly select overlay, tcp, or udp. Persisted rules without a protocol decode as overlay; new IPC mutations must specify one. A stream or text message is delivered only when an overlay allow rule matches its authenticated source and a local process currently binds the destination port. Outbound native flows are allowed, return TCP/UDP state is bounded and timed, and new inbound flows require a matching source/port rule. ICMPv6 echo has per-source rate limiting. Firewall rules affect terminating traffic, not transit forwarding.

Local tools use a mode-0600 Unix-domain socket. IPC messages use the same four-byte framing and binary property-list encoding, carry IPC version 2, and support control operations, interface diagnostics, service binding, and stream events. A binding is removed when its IPC connection closes.

Privileged interface protocol

The root LaunchDaemon accepts only the installing UID (verified with getpeereid) on its mode-0600 socket. After an UMN2 <local-address> greeting it creates an MTU-1280 utun, configures the local /128, and sends a duplicated descriptor using SCM_RIGHTS. Subsequent ROUTES messages replace a validated set of at most 31 mesh /128 routes. Commands are invoked as fixed /sbin/ifconfig or /sbin/route argument arrays, never through a shell. EOF removes all routes and closes the interface.