# Live STT WebSocket protocol -- `wss://.../v1/stream`

OpenAPI can't express WebSockets; this document is the contract for the
stt-proxy Worker's live-transcription relay. Implementation:
`workers/stt-proxy/src/index.js` + `translate.js` (keep in sync).
Consumers: the desktop's `WorkerProxyAdapter` (`src/main/agent/stt/`),
future React Native capture.

## Connect

```
wss://seameet-stt-proxy-dev.seameet.workers.dev/v1/stream?lang=<iso-639-1, optional>
Authorization: Bearer <supabase user JWT>
```

Auth is the standard `Authorization` header on the upgrade request --
Node/Electron and React Native WebSocket clients can set it; **browser
`WebSocket` cannot set headers**, so the web library does NOT do live STT
(by design -- capture is a device feature). Verification: ES256 against the
project JWKS with HS256 fallback. Entitlement: same `checkEntitlement` as
sync-api (plan `sync`, active/trialing, not ai_disabled); failures close
the socket immediately after upgrade.

The STT provider (Soniox `stt-rt-v4` / Deepgram `nova-3`) is chosen
server-side from the `provider_routing` table -- clients never pick. The
`lang` query hint participates in routing (`match_lang` rules).

## Client -> server frames

| Frame | Encoding | Meaning |
|---|---|---|
| Binary | PCM 16-bit LE, 16 kHz, mono | Audio; send in ~100-250 ms chunks |
| Text `{"type":"finalize"}` | JSON | Flush upstream + emit remaining finals; server closes when drained |

## Server -> client frames (JSON text)

| `type` | Payload | Meaning |
|---|---|---|
| `ready` | `provider` | Upstream relay connected; start sending audio |
| `segment` | `segment: TranscriptSegment` | One transcript segment. `segment.isFinal` distinguishes interim (replaces the previous interim) from committed text. Shape contract: `src/main/agent/stt/STTProvider.js` -- `{text, speaker?, language?, confidence?, startMs?, endMs?, isFinal}` |
| `error` | `error` (code string) | Fatal; socket closes next. Notable codes: `all_providers_unavailable` |

## Lifecycle & limits

- One session = one recording. Sessions are metered into `usage_events`
  (`feature: 'stt'`, `metadata.path: 'managed'`; Soniox sessions bill by
  the documented token conversion, Deepgram per second).
- Reconnect policy has two layers, and they are deliberately different speeds:
  - **Transport (`WorkerProxyAdapter`)**: one immediate reconnect on an
    unexpected mid-session close, invisible to the renderer.
  - **Session supervisor (renderer, `src/utils/sttReconnectPolicy.ts`)**: if the
    session is still dead after that, it retries while the recording is running
    and not paused, on a **slow** ladder -- 5 s / 15 s / 30 s, then 60 s. Slow by
    design: each attempt consumes a concurrency slot a zombie predecessor may
    still hold for its 120 s KV TTL, so fast retries make the outage worse. A
    `too_many_concurrent_sessions` rejection backs off a full 120 s (past the
    slot TTL). Terminal server states (free pool spent, 5 h cap) stop retrying.
    The BYOK Gemini Live path, which has no server slot accounting, uses the
    fast ladder instead (2/4/8/16 s, 30 s cap). The transcript panel also shows
    a manual **Reconnect** button whenever the session is `disconnected` or
    `error` and no hard wall is up; it is single-flight with the supervisor.
- Reconnect resumes as a new provider session -- no replay -- but the desktop
  keeps the existing transcript and re-anchors incoming timestamps to the
  recording's timeline (`sessionTimestampOffsetMs`), so captions stay
  continuous across a drop and across pause/resume rather than restarting at
  00:00.
- Audio is sent in ~250 ms batches (renderer resample worker,
  `BATCH_INTERVAL_MS`), not per AudioWorklet render quantum.
- Concurrency is capped per provider account and metered per uid by the proxy
  (KV slot, 120 s TTL, heartbeat-refreshed); `GET /v1/health` exposes live
  session counts (see `openapi.yaml`). The `/v1/file` upload lane shares the
  bucket, so a stuck slot also blocks the post-stop summary upload.