omnivox-rs/PROTOCOL-V2.md

OmniVox protocol v2.1

Wire-incompatible with the 2013 C++ OmniVox by design. Goals: perfect forward secrecy (ephemeral X25519), per-session per-direction keys, no IV-reuse hazard, replay defense, authenticated framing, and a stream lane for bulk data. Everything is one-shot AEAD — no streaming ciphers.

Handshake (OVX2)

TCP is connected, then both sides exchange a fixed 69-byte hello:

bytes 0..4     magic "OVX2"
byte  4        version 2
bytes 5..37    ephemeral X25519 public key
bytes 37..69   session nonce (32 random bytes)

A hello with wrong magic or version fails closed (connection closed, no response). Both hellos are exchanged before any key is derived, so the two peers are symmetric roles distinguished only by their nonces.

dh    = X25519(eph_sk_peer_side, eph_pk_other_side)
salt  = nonce_client ‖ nonce_server
prk   = HKDF-SHA256-Extract(salt, ikm = PSK ‖ dh)
send_key(server→client) = HKDF-Expand(prk, info = "omnivox2|c2s", 32)
send_key(client→server) = HKDF-Expand(prk, info = "omnivox2|s2c", 32)

(Names are from the client's view: each peer encrypts with the key whose name is the direction it writes.) The PSK is scrypt(password, salt, N=2^15, r=8, p=1, 32) — run on a blocking thread. The pre-hello exchange is unauthenticated; the first framed message (which authenticates under the derived keys) is the proof both sides hold the same PSK. An attacker without the PSK sees every frame fail authentication and the connection die — they never get an oracle, only a closed socket.

The ephemeral DH gives forward secrecy: recorded traffic cannot be decrypted later even if the PSK leaks. Session nonces bind keys to this run: frames captured from one session are ciphertext to another.

Frame format

Every post-handshake packet, in both directions:

u32 LE   len   total byte count of type ‖ ciphertext ‖ tag
u8       type  0 = message (bincode payload), 1 = stream (raw bytes)
bytes    ct    ChaCha20-Poly1305 ciphertext (plaintext length = len − 17)
bytes    tag   16-byte Poly1305 tag

len ∈ [17, max_frame]; zero or oversized is a protocol violation (close). max_frame is max_message_size for type 0 (default 64 KiB) and max_stream_frame for type 1 (default 1 MiB) — both caps are enforced before allocation.

Nonce

The nonce is not on the wire. It is derived from the sender's per-direction sequence number:

nonce(counter) = 00 00 00 00 ‖ u64 BE counter

Each sender's counter starts at 1 and increments by exactly one per frame; each receiver tracks its own expected counter, starting at 1. The receiver decrypts only with its expected counter, so a replayed or reordered frame authenticates against the wrong nonce and fails — guaranteed, not heuristic. 2^64 frames per direction before the nonce space is exhausted (close at the limit).

AAD

The type byte is the AEAD additional authenticated data. Flipping message↔stream is therefore an authentication failure, not a lane switch.

Failure policy

• Any frame-level failure (bad length, unknown type, AEAD failure) logs and closes the connection. v2.1 has no skip-and-continue: an authenticated bitstream that stops authenticating means someone is tampering or the two ends disagree about state — either way the session is over.
• Payload decode failure (frame authenticated, bincode decode fails, e.g. a Message claiming a 2^40 byte vector): the frame is dropped and counted, the connection stays open. Authenticated junk is the sender's bug, not an attack.
• Stream frames are handed to on_stream as raw bytes; no decoding.

Framing discipline

• A body-stall timer (body_stall_ms) fires while reading a frame body that started but never completes.
• An idle timer (idle_timeout_ms) closes connections with no inbound frames at all.
• The reader loop processes frames strictly in wire order; messages and stream chunks share one sequence, so callbacks observe the exact interleaving the sender produced.

Outbound path

Each connection owns a bounded queue (depth 8) and a writer task. send/send_stream await queue space (backpressure); the synchronous callback helpers reply/reply_stream/send_to use try_send and return false when full instead of blocking the reader lane. The writer task seals frames with its own counter and writes them out, so slow sockets never stall the read loop.

Timeouts (defaults)

Timer	Default	Fires when
handshake	10 s	hello exchange incomplete
body stall	5 s	frame body started, never finished
idle	15 s	no inbound frames at all
send queue	5 s	outbound queue full and staying full

Test coverage

Property	Mechanism	Test
Confidentiality	per-session per-direction keys	frames_from_another_session_are_rejected
Replay defense	derived nonce + expected counter	replayed_frame_kills_the_connection
Session binding	nonces in salt	same cross-session test
Type tampering	type byte as AAD	frame_type_tamper_is_rejected
Version downgrade	magic+version gate	wrong_version_hello_is_closed
v1 traffic	fails hello	v1_traffic_is_rejected_at_the_handshake
Handler panics	catch_unwind per callback	panicking_callbacks_never_kill_the_connection
Decode bombs	skip authenticated junk	lying_decoder_length_is_skipped_connection_survives
Silent peer	idle timers both roles	malicious_server_silence_times_out_the_client
Ordering	shared sequence	messages_and_streams_interleave_in_order