11 Commits

Author SHA1 Message Date
egutierrez e3f40913bc chore(deploy): version the same-origin Caddy config for uniweb
Capture the reverse-proxy vhost that fronts the browser-native uniweb
client on magnus (chat-<hash>.organic-machine.com): the SPA at /, the
signed control plane under /api (prefix stripped so request signatures
verify), and the NATS-over-WebSocket data plane under /nats. One origin
means no CORS and keeps the cluster node IPs hidden behind the proxy.

Self-contained fragment (includes the shared security_headers snippet) so
it validates with `caddy validate` on its own; the other vhosts on magnus
carry basic-auth secrets and are intentionally left out of git. Documents
the matching membershipd flags this config requires (--cors-origins with
the same-origin host, --trusted-proxies naming the Caddy node).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 12:43:15 +02:00
egutierrez 0b96c114b6 feat(membership): trust reverse-proxy forwarded client IP for rate limit
The per-IP rate limiter keys on the transport RemoteAddr. Behind the
same-origin Caddy proxy that fronts the control plane, every request
arrives with the proxy's single IP, which collapses the limiter into one
bucket shared by the whole world — a flood from one client throttles all
of them.

Add an opt-in `--trusted-proxies` flag (comma-separated IPs/CIDRs). When
the immediate peer is one of the named proxies, clientIP now believes its
X-Forwarded-For (read right-to-left, skipping trusted hops) or X-Real-IP
and keys on the real client. A direct, non-trusted peer's forwarding
headers are ignored entirely, so this opens no quota-fanning hole: an
attacker connecting straight to the public :8470 cannot spoof a key. The
zero value (no flag) preserves the prior RemoteAddr-only behavior exactly.

Covered by ratelimit_proxy_test.go: trusted vs untrusted peers, XFF
right-to-left precedence, client-prepended forgery, X-Real-IP fallback,
and rejection of malformed proxy entries.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 12:29:57 +02:00
egutierrez 294905984c fix(membership): allow X-Unibus-* auth headers in CORS preflight
A browser signs every control-plane request with X-Unibus-Pub/Ts/Nonce/Sig
(busauth.signedHeaders). The CORS Allow-Headers only listed Content-Type and
Authorization, so the browser's preflight rejected the real request and the SPA
failed with 'Failed to fetch' on the first authenticated call (listRooms). Add the
four X-Unibus-* headers to Access-Control-Allow-Headers.

This was invisible to the Node smoke (fetch in Node does no CORS preflight); only a
real browser surfaced it. Verified live: enmanuel logs into uniweb against the
cluster and lists rooms. Regression test asserts the header is present.
2026-06-14 12:12:20 +02:00
egutierrez feb917fc6a feat(cluster): deploy browser WebSocket + CORS to the 3-node cluster
Roll the --ws-port + --cors-origins flags (issue uniweb/0001) out to the unibus
cluster so the browser-native uniweb client can reach the data plane (nats.ws)
and the control plane (CORS) on every node. The WS reuses the data-plane TLS
(wss://) and the same origin allowlist.

Per-node WS port override (WS_PORT_<NAME>): magnus runs unibus_admin on
127.0.0.1:8480, so the bus WS binds 8485 there to avoid a crash-loop; homer and
datardos keep 8480. deploy-cluster.sh also gains DEPLOY_ONLY=<name> for rolling
one node at a time. Rolled out and verified 2026-06-13: all three nodes healthy,
WS reachable, CORS 204, cluster quorum (R3) intact throughout.
2026-06-13 23:23:52 +02:00
egutierrez c0216de766 feat(membershipd): --ws-port wires the embedded NATS WebSocket listener
Phase 0 left the WebsocketConfig field unwired; add --ws-port so membershipd can
actually expose the browser data-plane transport. It reuses the data-plane TLS
(wss:// when TLS is on, ws:// for a loopback dev stack) and the same
--cors-origins allowlist that gates the control plane, so one flag pair opens
both planes to the browser-native uniweb client (issue uniweb/0001).
2026-06-13 23:05:33 +02:00
egutierrez 0088fb946b feat(busvectors): add nkey + signed control-request vectors
Extend the cross-language vectors with the NATS user nkey derived from the
Ed25519 public key, and a signed control-plane request (CanonicalRequest +
Ed25519 signature). These let the TypeScript busauth port verify it authenticates
on both planes exactly like the Go client (issue uniweb/0001, Phase 1).
2026-06-13 22:49:20 +02:00
egutierrez e058b324f4 Merge branch 'quick/0001-ws-cors-prep' 2026-06-13 22:21:51 +02:00
egutierrez a5086ecd18 chore: bump unibus to 0.14.0 (browser-native client prep, Phase 0) 2026-06-13 22:21:51 +02:00
egutierrez 8a51c5cc1f feat(busvectors): deterministic cross-language test vectors
Add cmd/busvectors, a generator that emits stable JSON test vectors for the bus
protocol and its E2E crypto (endpoint id, Ed25519 sign, ChaCha20-Poly1305 AEAD
with a fixed nonce, sealed-box of a room key, and canonical Frame wire bytes +
SigningBytes). It uses the same registry crypto (functions/cybersecurity) the
bus uses, so the vectors are the contract the TypeScript port must match
byte-for-byte (issue uniweb/0001, Phase 0).

Regenerate with: go run ./cmd/busvectors > ../uniweb/web/src/bus/testdata/vectors.json
2026-06-13 22:21:32 +02:00
egutierrez ec8d34aaa1 feat(membership): opt-in CORS allowlist for the browser-native client
Add Server.AllowedOrigins and an applyCORS step at the top of ServeHTTP so a
browser SPA (uniweb) can call the control plane cross-origin: an allow-listed
Origin gets the Access-Control-Allow-* headers, and a preflight (OPTIONS) is
answered 204 before the rate limiter and auth ever run. A disallowed or missing
origin gets no headers (preflight 403), so the browser blocks the request.

Wire it through membershipd's --cors-origins flag (comma list, reusing
splitRoutes as a generic parser). Empty allowlist = CORS off, no headers
emitted, behavior identical to before: native Go/Kotlin clients send no Origin
and are unaffected. Opt-in per deployment (issue uniweb/0001, Phase 0).

Tests: preflight allow/deny, header on the real response, CORS-off default, and
no-Origin native client unaffected.
2026-06-13 22:17:44 +02:00
egutierrez 36f4ba0eaf feat(embeddednats): optional WebSocket listener for browser clients
Add WebsocketConfig to ServerConfig so the embedded nats-server can expose an
additional WebSocket port (nats.ws) alongside the TCP data plane. This lets a
browser SPA speak the NATS protocol directly, the way native TCP peers (Go,
Kotlin/android) already do — the first enabler for uniweb becoming a
browser-native client with no Go gateway (issue uniweb/0001, Phase 0).

The client authenticator applies to WebSocket connections too, so this adds a
transport, not a trust bypass. Plain ws:// is used only without TLS (loopback
dev); a certificate yields wss://. An empty AllowedOrigins enforces same-origin.
Nil WebsocketConfig keeps the server TCP-only, so existing single-node and
cluster deployments are unchanged.

Tests: WebSocket listener opens and completes the upgrade handshake (101); no
listener opens when WebsocketConfig is nil.
2026-06-13 22:11:39 +02:00
13 changed files with 1077 additions and 11 deletions
+11 -1
View File
@@ -2,7 +2,7 @@
name: unibus name: unibus
lang: go lang: go
domain: infra domain: infra
version: 0.13.0 version: 0.14.0
description: "Bus de mensajería unificado sobre NATS+JetStream con cifrado E2E por room (megolm/olm reducido): service de membresía/claves, librería cliente y peers demo." description: "Bus de mensajería unificado sobre NATS+JetStream con cifrado E2E por room (megolm/olm reducido): service de membresía/claves, librería cliente y peers demo."
tags: [service, messaging, nats, e2e] tags: [service, messaging, nats, e2e]
uses_functions: uses_functions:
@@ -169,6 +169,16 @@ agent.<nombre>.{in,out} inbox/outbox de agente LLM (agent.scout.in)
## Capability growth log ## Capability growth log
- v0.14.0 (2026-06-13) — prep para el cliente browser-nativo `uniweb` (issue
uniweb/0001, Fase 0), todo aditivo y opt-in: (1) el nats-server embebido puede
exponer un listener WebSocket (`WebsocketConfig`) para que un navegador hable el
protocolo NATS via `nats.ws`, igual que los peers TCP nativos; el authenticator
nkey aplica también al WebSocket. (2) El control-plane (`membershipd`) gana una
allowlist CORS opt-in (`--cors-origins`) para aceptar llamadas cross-origin del
navegador; vacía = CORS off, sin cambios para clientes nativos. (3) `cmd/busvectors`
genera vectores de test deterministas (endpoint id, firma Ed25519, AEAD
ChaCha20-Poly1305, sealed-box, wire del Frame) como contrato de paridad para el
port TypeScript. Peers Go/Kotlin existentes sin cambios; build/vet/test verdes.
- v0.13.0 (2026-06-13) — el frontend web se separa a su propia app `uniweb` - v0.13.0 (2026-06-13) — el frontend web se separa a su propia app `uniweb`
(`projects/message_bus/apps/uniweb`, sub-repo Gitea propio). unibus deja de (`projects/message_bus/apps/uniweb`, sub-repo Gitea propio). unibus deja de
contener la SPA (`web/`) y el gateway web (`cmd/webgw`): ahora es estrictamente contener la SPA (`web/`) y el gateway web (`cmd/webgw`): ahora es estrictamente
+281
View File
@@ -0,0 +1,281 @@
// Command busvectors emits deterministic cross-language test vectors for the bus
// protocol and its end-to-end crypto. The browser-native client (uniweb) ports the
// protocol to TypeScript; these vectors are the contract that proves the port is
// byte-for-byte compatible with this Go reference implementation (issue
// uniweb/0001, Phase 0).
//
// Every input is fixed (hardcoded key material and messages) so the output is
// stable across runs and can be committed as a golden file. The crypto primitives
// are the SAME registry functions the bus uses (functions/cybersecurity), so the
// vectors exercise the real path, not a test-only reimplementation.
//
// Coverage:
// - endpoint_id : EndpointID(signPub) = base64url(sha256(signPub))
// - sign : Ed25519 signature over a fixed message (deterministic)
// - aead : ChaCha20-Poly1305 seal with a FIXED nonce (deterministic, so
// the TS port must reproduce the same ciphertext AND open it)
// - keybox : sealed-box (X25519) of a room key for a recipient; the TS port
// must OPEN it (the ephemeral sender key is random, so only the
// open direction is a stable vector — the TS->Go seal direction
// is covered by the live E2E test in Phase 3)
// - frame : canonical JSON wire bytes of a Frame, and its SigningBytes
//
// Usage:
//
// go run ./cmd/busvectors > ../uniweb/web/src/bus/testdata/vectors.json
package main
import (
"crypto/ed25519"
"encoding/base64"
"encoding/hex"
"encoding/json"
"fmt"
"os"
cs "fn-registry/functions/cybersecurity"
"github.com/enmanuel/unibus/pkg/busauth"
"github.com/enmanuel/unibus/pkg/frame"
"github.com/enmanuel/unibus/pkg/membership"
"golang.org/x/crypto/chacha20poly1305"
"golang.org/x/crypto/curve25519"
)
// Fixed key material. The bytes are arbitrary but stable: the point is a golden
// file, not secrecy (these are test vectors, never real identities).
var (
signSeed = mustHex("000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f")
kexPriv = mustHex("202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f")
recipientKexPriv = mustHex("404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f")
aeadKey = mustHex("606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f")
aeadNonce = mustHex("808182838485868788898a8b") // 12 bytes (ChaCha20-Poly1305 IETF)
roomKey = mustHex("a0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebf")
signMessage = []byte("unibus parity vector message")
aeadAAD = []byte("unibus-room-42")
aeadPlaintext = []byte("hello from the bus")
)
// vectors is the JSON document consumed by the TypeScript parity tests. Every field
// is hex except the frame wire bytes, which are base64 (the frame is JSON, so the
// TS side compares the exact UTF-8 bytes).
type vectors struct {
Note string `json:"note"`
Endpoint endpointVector `json:"endpoint_id"`
Nkey nkeyVector `json:"nkey"`
Sign signVector `json:"sign"`
AEAD aeadVector `json:"aead"`
KeyBox keyboxVector `json:"keybox"`
Frame frameVector `json:"frame"`
CtrlReq controlReqVector `json:"control_request"`
}
type endpointVector struct {
SignPubHex string `json:"sign_pub_hex"`
EndpointID string `json:"endpoint_id"` // base64url(sha256(sign_pub)), unpadded
}
type nkeyVector struct {
SignPubHex string `json:"sign_pub_hex"`
NkeyPublic string `json:"nkey_public"` // NATS user nkey ("U...") from the Ed25519 pubkey
}
type controlReqVector struct {
Method string `json:"method"`
Path string `json:"path"`
Ts string `json:"ts"`
Nonce string `json:"nonce"`
BodyHex string `json:"body_hex"` // raw request body (empty for GET)
CanonicalHex string `json:"canonical_hex"` // bytes that get signed
SigHex string `json:"sig_hex"` // Ed25519 over canonical, by the signer below
SignPrivHex string `json:"sign_priv_hex"`
}
type signVector struct {
SignPrivHex string `json:"sign_priv_hex"`
SignPubHex string `json:"sign_pub_hex"`
MessageHex string `json:"message_hex"`
SigHex string `json:"sig_hex"`
}
type aeadVector struct {
KeyHex string `json:"key_hex"`
NonceHex string `json:"nonce_hex"`
AADHex string `json:"aad_hex"`
PlaintextHex string `json:"plaintext_hex"`
CiphertextHex string `json:"ciphertext_hex"` // includes the 16-byte Poly1305 tag
}
type keyboxVector struct {
RecipientKexPubHex string `json:"recipient_kex_pub_hex"`
RecipientKexPrivHex string `json:"recipient_kex_priv_hex"`
SecretHex string `json:"secret_hex"`
SealedHex string `json:"sealed_hex"`
}
type frameVector struct {
// The source fields, so the TS side can build the same Frame and compare.
Type int `json:"type"`
Subject string `json:"subject"`
Sender string `json:"sender"`
MsgID string `json:"msg_id"`
Epoch int `json:"epoch"`
NonceHex string `json:"nonce_hex"`
PayloadHex string `json:"payload_hex"`
WireB64 string `json:"wire_b64"` // base64(Marshal()) — full frame incl. sig
SigningB64 string `json:"signing_bytes_b64"` // base64(SigningBytes()) — what gets signed
SigHex string `json:"sig_hex"` // Ed25519 over SigningBytes
}
func main() {
if err := run(os.Stdout); err != nil {
fmt.Fprintln(os.Stderr, "busvectors:", err)
os.Exit(1)
}
}
func run(out *os.File) error {
// Identity from the fixed seed: Go's ed25519 private key layout is seed||pub, the
// same 64-byte layout cs.Identity and the TS wallet use.
signPriv := ed25519.NewKeyFromSeed(signSeed)
signPub := signPriv.Public().(ed25519.PublicKey)
// X25519 public keys from the fixed private scalars (curve25519 clamps internally,
// matching @noble/curves x25519.getPublicKey).
kexPub, err := curve25519.X25519(kexPriv, curve25519.Basepoint)
if err != nil {
return fmt.Errorf("kex pub: %w", err)
}
recipientKexPub, err := curve25519.X25519(recipientKexPriv, curve25519.Basepoint)
if err != nil {
return fmt.Errorf("recipient kex pub: %w", err)
}
// AEAD with a FIXED nonce so the vector is deterministic. This is the same cipher
// (ChaCha20-Poly1305 IETF, 12-byte nonce) that cs.SealAEAD uses; we set the nonce
// explicitly only to make the vector reproducible. OpenAEAD verifies round-trip.
aead, err := chacha20poly1305.New(aeadKey)
if err != nil {
return fmt.Errorf("aead cipher: %w", err)
}
ciphertext := aead.Seal(nil, aeadNonce, aeadPlaintext, aeadAAD)
if _, err := cs.OpenAEAD(aeadKey, aeadNonce, ciphertext, aeadAAD); err != nil {
return fmt.Errorf("aead self-check: %w", err)
}
// Sealed box of the room key for the recipient. The sender's ephemeral key is
// random (anonymous sealed box), so SealedHex changes per run; the stable, useful
// assertion for the TS port is that OpenKeyBox recovers the secret, which we
// self-check here. The TS test opens SealedHex and compares to SecretHex.
sealed, err := cs.SealKeyBox(recipientKexPub, roomKey)
if err != nil {
return fmt.Errorf("seal keybox: %w", err)
}
if got, err := cs.OpenKeyBox(recipientKexPub, recipientKexPriv, sealed); err != nil || hex.EncodeToString(got) != hex.EncodeToString(roomKey) {
return fmt.Errorf("keybox self-check failed: %v", err)
}
// A representative encrypted-room frame, signed end-to-end.
f := frame.Frame{
Type: frame.PUB,
Subject: "room.parity",
Sender: frame.EndpointID(signPub),
MsgID: "01HZY0VECTORFIXEDULID0001",
Epoch: 1,
Nonce: aeadNonce,
Payload: ciphertext,
}
f.Sig = ed25519.Sign(signPriv, f.SigningBytes())
wire, err := f.Marshal()
if err != nil {
return fmt.Errorf("marshal frame: %w", err)
}
// NATS user nkey derived from the Ed25519 public key (the browser must produce
// the same "U..." string to authenticate on the data plane).
nkeyPub, err := busauth.NkeyPublicFromSignPub(signPub)
if err != nil {
return fmt.Errorf("nkey public: %w", err)
}
// A signed control-plane request vector: the browser signs CanonicalRequest the
// same way to authenticate every HTTP call to membershipd. A POST with a body
// exercises the sha256(body) term.
const ctrlMethod = "POST"
const ctrlPath = "/rooms"
const ctrlTs = "1700000000"
const ctrlNonce = "Zm9vYmFyMTIzNDU2Nzg5MA=="
ctrlBody := []byte(`{"subject":"room.parity"}`)
canonical := membership.CanonicalRequest(ctrlMethod, ctrlPath, ctrlTs, ctrlNonce, ctrlBody)
ctrlSig := ed25519.Sign(signPriv, canonical)
v := vectors{
Note: "Deterministic cross-language vectors for the unibus protocol. Generated by " +
"cmd/busvectors in the unibus repo; regenerate with `go run ./cmd/busvectors`. " +
"sealed_hex varies per run (anonymous sealed box); assert via OpenKeyBox.",
Endpoint: endpointVector{
SignPubHex: hex.EncodeToString(signPub),
EndpointID: frame.EndpointID(signPub),
},
Nkey: nkeyVector{
SignPubHex: hex.EncodeToString(signPub),
NkeyPublic: nkeyPub,
},
Sign: signVector{
SignPrivHex: hex.EncodeToString(signPriv),
SignPubHex: hex.EncodeToString(signPub),
MessageHex: hex.EncodeToString(signMessage),
SigHex: hex.EncodeToString(ed25519.Sign(signPriv, signMessage)),
},
AEAD: aeadVector{
KeyHex: hex.EncodeToString(aeadKey),
NonceHex: hex.EncodeToString(aeadNonce),
AADHex: hex.EncodeToString(aeadAAD),
PlaintextHex: hex.EncodeToString(aeadPlaintext),
CiphertextHex: hex.EncodeToString(ciphertext),
},
KeyBox: keyboxVector{
RecipientKexPubHex: hex.EncodeToString(recipientKexPub),
RecipientKexPrivHex: hex.EncodeToString(recipientKexPriv),
SecretHex: hex.EncodeToString(roomKey),
SealedHex: hex.EncodeToString(sealed),
},
Frame: frameVector{
Type: int(f.Type),
Subject: f.Subject,
Sender: f.Sender,
MsgID: f.MsgID,
Epoch: f.Epoch,
NonceHex: hex.EncodeToString(f.Nonce),
PayloadHex: hex.EncodeToString(f.Payload),
WireB64: base64.StdEncoding.EncodeToString(wire),
SigningB64: base64.StdEncoding.EncodeToString(f.SigningBytes()),
SigHex: hex.EncodeToString(f.Sig),
},
CtrlReq: controlReqVector{
Method: ctrlMethod,
Path: ctrlPath,
Ts: ctrlTs,
Nonce: ctrlNonce,
BodyHex: hex.EncodeToString(ctrlBody),
CanonicalHex: hex.EncodeToString(canonical),
SigHex: hex.EncodeToString(ctrlSig),
SignPrivHex: hex.EncodeToString(signPriv),
},
// kexPub is unused in a vector field today but derived above to validate the
// scalar; reference it so the intent is documented.
}
_ = kexPub
enc := json.NewEncoder(out)
enc.SetIndent("", " ")
return enc.Encode(v)
}
func mustHex(s string) []byte {
b, err := hex.DecodeString(s)
if err != nil {
panic("busvectors: bad fixed hex: " + s)
}
return b
}
+40
View File
@@ -13,6 +13,7 @@ import (
"net/http" "net/http"
"os" "os"
"os/signal" "os/signal"
"strings"
"syscall" "syscall"
"time" "time"
@@ -54,8 +55,11 @@ func main() {
dbPath = flag.String("db", "./local_files/unibus.db", "SQLite database path") dbPath = flag.String("db", "./local_files/unibus.db", "SQLite database path")
storeDir = flag.String("store-dir", "./local_files/blobs", "blob store directory") storeDir = flag.String("store-dir", "./local_files/blobs", "blob store directory")
natsPort = flag.Int("nats-port", 4250, "embedded NATS listen port (when --nats-url empty)") natsPort = flag.Int("nats-port", 4250, "embedded NATS listen port (when --nats-url empty)")
wsPort = flag.Int("ws-port", 0, "WebSocket listen port for browser clients (nats.ws); 0 = disabled. Enables the browser-native uniweb client (issue uniweb/0001)")
natsStore = flag.String("nats-store", "./local_files/jetstream", "embedded JetStream store dir") natsStore = flag.String("nats-store", "./local_files/jetstream", "embedded JetStream store dir")
busAuth = flag.String("bus-auth", "off", "control-plane auth rollout: off|soft|enforce (feature flag bus-auth)") busAuth = flag.String("bus-auth", "off", "control-plane auth rollout: off|soft|enforce (feature flag bus-auth)")
corsOrigins = flag.String("cors-origins", "", "comma-separated CORS allowlist of browser origins permitted to call the control plane (e.g. http://localhost:5173,https://chat.example.com); empty = CORS off. Enables the browser-native uniweb client (issue uniweb/0001)")
trustedProxies = flag.String("trusted-proxies", "", "comma-separated IPs/CIDRs of reverse proxies whose X-Forwarded-For/X-Real-IP is trusted for the per-IP rate limit; empty = trust the direct connection only. Set to the same-origin proxy's address (e.g. the Caddy node) so the rate limit stays per-client behind the proxy")
tlsCert = flag.String("tls-cert", "", "PATH to the NATS server certificate (deploy/tls/server.crt); enables TLS on the embedded data plane") tlsCert = flag.String("tls-cert", "", "PATH to the NATS server certificate (deploy/tls/server.crt); enables TLS on the embedded data plane")
tlsKey = flag.String("tls-key", "", "path to the NATS server private key (deploy/tls/server.key); required with --tls-cert") tlsKey = flag.String("tls-key", "", "path to the NATS server private key (deploy/tls/server.key); required with --tls-cert")
// Cluster (issue 0003a): empty --cluster-name keeps the server standalone. // Cluster (issue 0003a): empty --cluster-name keeps the server standalone.
@@ -267,6 +271,24 @@ func main() {
cfg.TLS = tlsCfg cfg.TLS = tlsCfg
log.Printf("NATS TLS: ON (%s)", *tlsCert) log.Printf("NATS TLS: ON (%s)", *tlsCert)
} }
if *wsPort > 0 {
// Expose a WebSocket listener so browser clients (uniweb via nats.ws) reach
// the data plane directly. It reuses the data-plane TLS (wss:// when TLS is
// on, ws:// for a loopback dev stack) and the same browser-origin allowlist
// as the control-plane CORS, so opening the data plane to the browser and
// opening the control plane to it are governed by one --cors-origins list.
scheme := "ws"
if cfg.TLS != nil {
scheme = "wss"
}
cfg.Websocket = &embeddednats.WebsocketConfig{
Host: *bind,
Port: *wsPort,
TLS: cfg.TLS,
AllowedOrigins: splitRoutes(*corsOrigins),
}
log.Printf("NATS WebSocket: ON (%s://%s:%d)", scheme, *bind, *wsPort)
}
ns, err = embeddednats.StartServer(cfg) ns, err = embeddednats.StartServer(cfg)
if err != nil { if err != nil {
log.Fatalf("start embedded nats: %v", err) log.Fatalf("start embedded nats: %v", err)
@@ -329,6 +351,24 @@ func main() {
Cluster: clustered, Cluster: clustered,
Store: *storeBackend, Store: *storeBackend,
} }
// CORS allowlist for the browser-native client (uniweb). splitRoutes is reused
// as a generic comma-list parser (trim + drop empties). Empty flag => empty
// slice => CORS stays off, identical to the pre-flag behavior.
if origins := splitRoutes(*corsOrigins); len(origins) > 0 {
srv.AllowedOrigins = origins
log.Printf("CORS: allowing %d browser origin(s): %s", len(origins), strings.Join(origins, ", "))
}
// Trusted reverse proxies for the per-IP rate limit. Behind the same-origin
// Caddy proxy every request arrives with the proxy's IP, which would collapse
// the per-IP rate limit into one bucket for the whole world; naming the proxy
// here lets the limiter believe its X-Forwarded-For and key on the real client
// instead. Empty flag => trust the direct connection only (unchanged behavior).
if proxies := splitRoutes(*trustedProxies); len(proxies) > 0 {
if err := srv.SetTrustedProxies(proxies); err != nil {
log.Fatalf("invalid --trusted-proxies: %v", err)
}
log.Printf("rate limit: trusting forwarded client IP from proxies: %s", strings.Join(proxies, ", "))
}
// Replicated anti-replay (issue 0006a, audit 0008 N3): a clustered node MUST // Replicated anti-replay (issue 0006a, audit 0008 N3): a clustered node MUST
// share its nonce store across the cluster, or a request accepted on one node // share its nonce store across the cluster, or a request accepted on one node
+84
View File
@@ -0,0 +1,84 @@
# Same-origin reverse proxy for the browser-native uniweb chat client.
#
# This is the self-contained fragment that exposes uniweb on magnus
# (organic-machine.com). It is merged into magnus's /etc/caddy/Caddyfile, which
# also hosts unrelated services; only this service's blocks are versioned here
# (the other vhosts carry basic-auth secrets that do not belong in git). The live
# file imports the shared (security_headers) snippet that is duplicated below so
# this fragment validates on its own.
#
# One origin fronts the whole app so the SPA and the bus share an origin: no CORS,
# and the unibus cluster node IPs stay hidden behind this proxy. Caddy obtains and
# renews the Let's Encrypt certificate automatically (the *.organic-machine.com
# wildcard A record points here).
#
# / -> the static SPA (uniweb web/dist) with a single-page-app fallback
# /api/* -> the signed HTTPS control plane (membershipd :8470), prefix stripped
# /nats -> the NATS-over-WebSocket data plane (:8485 magnus / :8480 peers)
#
# Upstreams speak TLS with the bus's own self-signed CA, so Caddy skips upstream
# verification (the hop is still encrypted). The control plane signs requests over
# the UNPREFIXED path, so /api MUST be stripped (handle_path) or signatures fail.
#
# The membershipd nodes must run with the same-origin host in --cors-origins (so
# the NATS WebSocket Origin check accepts it) and with --trusted-proxies naming
# this Caddy node (127.0.0.1,::1,135.125.201.30) so the per-IP rate limit keys on
# the real client behind the proxy instead of collapsing to the proxy's one IP.
(security_headers) {
header {
Strict-Transport-Security "max-age=31536000"
X-Content-Type-Options "nosniff"
X-Frame-Options "DENY"
Referrer-Policy "no-referrer"
-Server
}
}
chat-c200aa64c3125ce8b5f068e0.organic-machine.com {
import security_headers
# Control plane: strip /api so /api/rooms reaches membershipd as /rooms (the
# path the client signs). Prefer the local node; lb_try_duration retries the
# next node within the request on a dial error (safe: a refused connection sent
# no bytes, so even a POST cannot double-apply), and fail_duration plus the
# active /healthz check take a down node out of rotation.
handle_path /api/* {
reverse_proxy https://127.0.0.1:8470 https://141.94.69.66:8470 https://51.91.100.142:8470 {
transport http {
tls_insecure_skip_verify
}
lb_policy first
lb_try_duration 5s
lb_try_interval 250ms
fail_duration 10s
health_uri /healthz
health_interval 10s
health_timeout 5s
}
}
# Data plane: NATS over WebSocket. Strip /nats so the upgrade reaches the ws
# listener at its root. Caddy proxies the WebSocket upgrade natively. The ws
# listener speaks TLS on :8485 (magnus; :8480 is taken by unibus_admin there)
# and :8480 on the peers. Passive failover only (an HTTP health probe would be
# rejected by the NATS ws endpoint).
handle_path /nats* {
reverse_proxy https://127.0.0.1:8485 https://141.94.69.66:8480 https://51.91.100.142:8480 {
transport http {
tls_insecure_skip_verify
}
lb_policy first
lb_try_duration 5s
lb_try_interval 250ms
fail_duration 30s
}
}
# SPA: static files with a client-side-routing fallback to index.html.
handle {
root * /opt/uniweb/dist
try_files {path} /index.html
file_server
}
}
+15
View File
@@ -69,6 +69,12 @@ routes_for() {
echo "==> [2/3] stage each node (REMOTE_DIR=$REMOTE_DIR)" echo "==> [2/3] stage each node (REMOTE_DIR=$REMOTE_DIR)"
for row in "${CLUSTER_NODES[@]}"; do for row in "${CLUSTER_NODES[@]}"; do
read -r name ssh _pub _wg <<<"$row" read -r name ssh _pub _wg <<<"$row"
# Rolling deploy: DEPLOY_ONLY=<name> stages just that node, so a new binary can be
# rolled out one node at a time (the other nodes keep the cluster quorum). Empty =
# stage every node (the original behavior).
if [[ -n "${DEPLOY_ONLY:-}" && "$name" != "$DEPLOY_ONLY" ]]; then
continue
fi
target="${SSH_USER}@${ssh}" target="${SSH_USER}@${ssh}"
nodedir="out/${name}" nodedir="out/${name}"
if [[ ! -d "$nodedir" ]]; then if [[ ! -d "$nodedir" ]]; then
@@ -79,6 +85,13 @@ for row in "${CLUSTER_NODES[@]}"; do
echo "-- node ${name} (ssh ${ssh}) routes=${routes}" echo "-- node ${name} (ssh ${ssh}) routes=${routes}"
# Resolve this node's WebSocket port. magnus runs unibus_admin on 127.0.0.1:8480,
# so the bus WS cannot bind 0.0.0.0:8480 there (it crash-loops). A per-node
# override (WS_PORT_<NAME> in nodes.env) lets magnus use a free port while the
# rest share the default — keeping the deploy reproducible (issue uniweb/0001).
node_ws_var="WS_PORT_${name^^}"
node_ws="${!node_ws_var:-$WS_PORT}"
# Generate this node's cluster.env locally, then ship it. # Generate this node's cluster.env locally, then ship it.
envfile="build/cluster-${name}.env" envfile="build/cluster-${name}.env"
mkdir -p build mkdir -p build
@@ -90,6 +103,8 @@ KV_REPLICAS=${KV_REPLICAS}
HTTP_PORT=${HTTP_PORT} HTTP_PORT=${HTTP_PORT}
NATS_CLIENT_PORT=${NATS_CLIENT_PORT} NATS_CLIENT_PORT=${NATS_CLIENT_PORT}
NATS_ROUTE_PORT=${NATS_ROUTE_PORT} NATS_ROUTE_PORT=${NATS_ROUTE_PORT}
WS_PORT=${node_ws}
CORS_ORIGINS=${CORS_ORIGINS}
ROUTES=${routes} ROUTES=${routes}
CLUSTER_PASS_FILE=${REMOTE_DIR}/secrets/cluster.pass CLUSTER_PASS_FILE=${REMOTE_DIR}/secrets/cluster.pass
TLS_CERT=${REMOTE_DIR}/tls/server-${name}.crt TLS_CERT=${REMOTE_DIR}/tls/server-${name}.crt
+3 -1
View File
@@ -35,7 +35,9 @@ ExecStart=/opt/unibus/membershipd \
--route-tls-ca ${ROUTE_TLS_CA} \ --route-tls-ca ${ROUTE_TLS_CA} \
--internal-id-file ${INTERNAL_ID_FILE} \ --internal-id-file ${INTERNAL_ID_FILE} \
--store kv \ --store kv \
--kv-replicas ${KV_REPLICAS} --kv-replicas ${KV_REPLICAS} \
--ws-port ${WS_PORT} \
--cors-origins ${CORS_ORIGINS}
# Restart=always (NOT on-failure): a clean SIGTERM exits success, and on-failure # Restart=always (NOT on-failure): a clean SIGTERM exits success, and on-failure
# would then NOT restart, leaving the node silently dead (see function_tags.md). # would then NOT restart, leaving the node silently dead (see function_tags.md).
Restart=always Restart=always
+13
View File
@@ -23,6 +23,19 @@ NATS_CLIENT_PORT=4250
NATS_ROUTE_PORT=6250 NATS_ROUTE_PORT=6250
HTTP_PORT=8470 HTTP_PORT=8470
# Browser data-plane: WebSocket listener so the browser-native uniweb client
# (nats.ws) reaches NATS, and the CORS allowlist for its calls to the control
# plane. WS reuses the data-plane TLS, so it serves wss:// (the cluster runs with
# TLS). CORS_ORIGINS is a comma-separated list of allowed browser origins (no
# spaces). Issue uniweb/0001. The node's firewall must allow WS_PORT.
WS_PORT=8480
# Per-node WS port override (WS_PORT_<NAME>). magnus runs unibus_admin on
# 127.0.0.1:8480, so the bus WebSocket cannot bind 0.0.0.0:8480 there — it would
# crash-loop. magnus therefore serves the browser WS on 8485; homer and datardos
# keep 8480 (no admin panel). Verified during the 2026-06-13 rollout.
WS_PORT_MAGNUS=8485
CORS_ORIGINS="http://localhost:5173"
# Remote install layout and SSH login user. # Remote install layout and SSH login user.
REMOTE_DIR="/opt/unibus" REMOTE_DIR="/opt/unibus"
SSH_USER="root" SSH_USER="root"
+59
View File
@@ -79,6 +79,42 @@ type ServerConfig struct {
// availability (issue 0003a). Nil keeps the server standalone (the legacy // availability (issue 0003a). Nil keeps the server standalone (the legacy
// single-node behavior). // single-node behavior).
Cluster *ClusterConfig Cluster *ClusterConfig
// Websocket, when non-nil, opens an ADDITIONAL WebSocket listener on the
// embedded nats-server so browser clients (nats.ws) can reach the data plane
// directly, the same way native TCP peers (Go, Kotlin) do (issue uniweb/0001).
// Native TCP clients are unaffected: the WebSocket listener is a separate port
// layered on top of the existing TCP listener, and the client authenticator
// (Auth) applies to both. Nil keeps the server TCP-only (legacy behavior).
Websocket *WebsocketConfig
}
// WebsocketConfig configures the embedded nats-server's WebSocket listener so a
// browser can speak the NATS protocol over ws://. A browser cannot open a raw TCP
// socket, so this is the only way the SPA reaches the data plane without a Go
// gateway in between.
//
// Security: off loopback a browser requires wss:// (TLS) — set TLS with a
// certificate the browser trusts. NoTLS plain ws:// is acceptable only for a
// loopback dev stack. The WebSocket upgrade also enforces an Origin allowlist
// (browser same-origin policy); AllowedOrigins must list the SPA's origins or the
// browser handshake is refused.
type WebsocketConfig struct {
// Host is the bind interface for the WebSocket listener; "" lets nats-server
// pick its default. Use "127.0.0.1" to keep it loopback-only in dev.
Host string
// Port is the WebSocket listen port (e.g. 8480). Required (non-zero) for the
// listener to open.
Port int
// NoTLS serves plain ws:// instead of wss://. Loopback/dev only: browsers refuse
// ws:// to a non-loopback origin. Ignored when TLS is set (TLS implies wss://).
NoTLS bool
// TLS, when set, serves wss:// with this certificate. Required for any browser
// origin that is not loopback.
TLS *tls.Config
// AllowedOrigins is the allowlist of browser Origin headers permitted to upgrade
// the WebSocket. Empty = same-origin only (nats-server SameOrigin). Never use a
// wildcard in production; list the exact SPA origins.
AllowedOrigins []string
} }
// Start is a thin backward-compatible wrapper: embedded JetStream server on the // Start is a thin backward-compatible wrapper: embedded JetStream server on the
@@ -170,6 +206,29 @@ func StartServer(cfg ServerConfig) (*server.Server, error) {
opts.TLS = true opts.TLS = true
} }
if cfg.Websocket != nil {
// Layer a WebSocket listener on top of the TCP data plane so browser
// clients (nats.ws) can connect. The client authenticator (opts.*Auth above)
// applies to WebSocket connections too, so a browser still has to pass the
// nkey + allowlist check; this only adds a transport, not a trust bypass.
ws := server.WebsocketOpts{
Host: cfg.Websocket.Host,
Port: cfg.Websocket.Port,
AllowedOrigins: cfg.Websocket.AllowedOrigins,
}
if cfg.Websocket.TLS != nil {
ws.TLSConfig = cfg.Websocket.TLS
} else {
// No certificate: plain ws:// (loopback/dev only). Browsers refuse this
// off-loopback, which is the intended guard rail.
ws.NoTLS = true
}
// Empty AllowedOrigins means "same-origin only": tell nats-server to enforce
// it rather than defaulting to accept-any-origin.
ws.SameOrigin = len(cfg.Websocket.AllowedOrigins) == 0
opts.Websocket = ws
}
if cfg.Cluster != nil { if cfg.Cluster != nil {
if err := applyClusterOpts(opts, cfg.Cluster); err != nil { if err := applyClusterOpts(opts, cfg.Cluster); err != nil {
return nil, err return nil, err
+108
View File
@@ -0,0 +1,108 @@
package embeddednats_test
import (
"fmt"
"net"
"net/http"
"strings"
"testing"
"time"
"github.com/enmanuel/unibus/pkg/embeddednats"
)
// wsFreePort returns an OS-assigned free TCP port on loopback. Kept local to this
// file so the WebSocket tests do not depend on the cluster test helpers.
func wsFreePort(t *testing.T) int {
t.Helper()
l, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("reserve free port: %v", err)
}
defer l.Close()
return l.Addr().(*net.TCPAddr).Port
}
// TestWebsocketListenerOpens verifies that when a ServerConfig carries a
// WebsocketConfig the embedded nats-server opens the additional WebSocket port and
// accepts a connection there, while the regular TCP client port keeps working. A
// browser cannot speak raw TCP, so this WebSocket listener is the only path the SPA
// has to the data plane (issue uniweb/0001).
func TestWebsocketListenerOpens(t *testing.T) {
clientPort := wsFreePort(t)
wsPort := wsFreePort(t)
ns, err := embeddednats.StartServer(embeddednats.ServerConfig{
StoreDir: t.TempDir(),
Host: "127.0.0.1",
Port: clientPort,
Websocket: &embeddednats.WebsocketConfig{
Host: "127.0.0.1",
Port: wsPort,
NoTLS: true, // loopback dev: plain ws://
},
})
if err != nil {
t.Fatalf("StartServer with websocket: %v", err)
}
t.Cleanup(func() { ns.Shutdown(); ns.WaitForShutdown() })
// The WebSocket listener must accept a TCP connection on its dedicated port.
addr := fmt.Sprintf("127.0.0.1:%d", wsPort)
conn, err := net.DialTimeout("tcp", addr, 2*time.Second)
if err != nil {
t.Fatalf("websocket port %d not accepting connections: %v", wsPort, err)
}
conn.Close()
// And it must speak the WebSocket upgrade handshake: a GET with the upgrade
// headers should get a 101 Switching Protocols (nats-server's ws endpoint),
// proving it is a real WebSocket listener, not just an open socket.
req, err := http.NewRequest(http.MethodGet, "http://"+addr+"/", nil)
if err != nil {
t.Fatalf("build upgrade request: %v", err)
}
req.Header.Set("Upgrade", "websocket")
req.Header.Set("Connection", "Upgrade")
req.Header.Set("Sec-WebSocket-Version", "13")
req.Header.Set("Sec-WebSocket-Key", "dGhlIHNhbXBsZSBub25jZQ==")
client := &http.Client{Timeout: 2 * time.Second}
resp, err := client.Do(req)
if err != nil {
t.Fatalf("websocket upgrade request: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusSwitchingProtocols {
t.Fatalf("websocket upgrade: got status %d, want 101 Switching Protocols", resp.StatusCode)
}
}
// TestNoWebsocketByDefault verifies the listener stays TCP-only when WebsocketConfig
// is nil: opening the browser transport must be an explicit opt-in so existing
// single-node and cluster deployments are unchanged.
func TestNoWebsocketByDefault(t *testing.T) {
clientPort := wsFreePort(t)
// Reserve a port, then free it, so we can assert nothing is listening there.
maybeWSPort := wsFreePort(t)
ns, err := embeddednats.StartServer(embeddednats.ServerConfig{
StoreDir: t.TempDir(),
Host: "127.0.0.1",
Port: clientPort,
// Websocket intentionally nil.
})
if err != nil {
t.Fatalf("StartServer: %v", err)
}
t.Cleanup(func() { ns.Shutdown(); ns.WaitForShutdown() })
conn, err := net.DialTimeout("tcp", fmt.Sprintf("127.0.0.1:%d", maybeWSPort), 300*time.Millisecond)
if err == nil {
conn.Close()
t.Fatalf("a listener is unexpectedly open on %d with no WebsocketConfig", maybeWSPort)
}
if !strings.Contains(err.Error(), "refused") && !strings.Contains(err.Error(), "timeout") {
t.Logf("dial error (acceptable, port closed): %v", err)
}
}
+157
View File
@@ -0,0 +1,157 @@
package membership_test
import (
"net/http"
"net/http/httptest"
"path/filepath"
"strings"
"testing"
"github.com/enmanuel/unibus/pkg/blobstore"
"github.com/enmanuel/unibus/pkg/membership"
)
// newCORSServer builds a control-plane server with the given CORS allowlist over a
// throwaway store, and returns a live httptest server. /healthz is auth-exempt, so
// the CORS tests can exercise the cross-origin pipeline without signing requests.
func newCORSServer(t *testing.T, origins ...string) *httptest.Server {
t.Helper()
dir := t.TempDir()
store, err := membership.Open(filepath.Join(dir, "unibus.db"))
if err != nil {
t.Fatalf("store: %v", err)
}
t.Cleanup(func() { store.Close() })
blobs, _ := blobstore.New(filepath.Join(dir, "blobs"))
srv := membership.NewServer(store, blobs, membership.AuthOff)
srv.AllowedOrigins = origins
ts := httptest.NewServer(srv)
t.Cleanup(ts.Close)
return ts
}
// TestCORSPreflightAllowedOrigin: a preflight (OPTIONS) from an allow-listed origin
// is answered 204 with the Access-Control headers, and never reaches auth. This is
// what lets the browser-native uniweb client call the control plane (issue
// uniweb/0001).
func TestCORSPreflightAllowedOrigin(t *testing.T) {
const origin = "http://localhost:5173"
ts := newCORSServer(t, origin)
req, _ := http.NewRequest(http.MethodOptions, ts.URL+"/rooms", nil)
req.Header.Set("Origin", origin)
req.Header.Set("Access-Control-Request-Method", "POST")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("preflight: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusNoContent {
t.Fatalf("preflight status = %d, want 204", resp.StatusCode)
}
if got := resp.Header.Get("Access-Control-Allow-Origin"); got != origin {
t.Fatalf("Allow-Origin = %q, want %q", got, origin)
}
if got := resp.Header.Get("Access-Control-Allow-Methods"); got == "" {
t.Fatalf("Allow-Methods missing on preflight")
}
// The control-plane request-auth headers a browser signs every request with must
// be allow-listed, or the browser's preflight blocks the real request (the bug a
// live browser surfaced: listRooms failed with "Failed to fetch").
if got := resp.Header.Get("Access-Control-Allow-Headers"); !strings.Contains(got, "X-Unibus-Sig") {
t.Fatalf("Allow-Headers must include the X-Unibus-* auth headers, got %q", got)
}
}
// TestCORSPreflightDisallowedOrigin: a preflight from an origin NOT in the allowlist
// gets 403 and no Access-Control headers, so the browser blocks the real request.
func TestCORSPreflightDisallowedOrigin(t *testing.T) {
ts := newCORSServer(t, "http://localhost:5173")
req, _ := http.NewRequest(http.MethodOptions, ts.URL+"/rooms", nil)
req.Header.Set("Origin", "https://evil.example.com")
req.Header.Set("Access-Control-Request-Method", "POST")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("preflight: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusForbidden {
t.Fatalf("disallowed preflight status = %d, want 403", resp.StatusCode)
}
if got := resp.Header.Get("Access-Control-Allow-Origin"); got != "" {
t.Fatalf("Allow-Origin leaked for disallowed origin: %q", got)
}
}
// TestCORSActualRequestCarriesHeader: a real GET from an allow-listed origin is
// served normally AND carries the Allow-Origin header so the browser accepts the
// response.
func TestCORSActualRequestCarriesHeader(t *testing.T) {
const origin = "http://localhost:5173"
ts := newCORSServer(t, origin)
req, _ := http.NewRequest(http.MethodGet, ts.URL+"/healthz", nil)
req.Header.Set("Origin", origin)
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("get: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("healthz status = %d, want 200", resp.StatusCode)
}
if got := resp.Header.Get("Access-Control-Allow-Origin"); got != origin {
t.Fatalf("Allow-Origin = %q, want %q", got, origin)
}
}
// TestCORSDisabledByDefault: with an empty allowlist no Access-Control header is
// ever emitted (CORS off) and requests behave exactly as before. This guards the
// opt-in invariant: untouched deployments are unaffected.
func TestCORSDisabledByDefault(t *testing.T) {
ts := newCORSServer(t) // no origins
req, _ := http.NewRequest(http.MethodGet, ts.URL+"/healthz", nil)
req.Header.Set("Origin", "http://localhost:5173")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("get: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("healthz status = %d, want 200", resp.StatusCode)
}
if got := resp.Header.Get("Access-Control-Allow-Origin"); got != "" {
t.Fatalf("Allow-Origin emitted with CORS off: %q", got)
}
}
// TestCORSNativeClientUnaffected: a request with no Origin header (a native Go/Kotlin
// client) is processed normally and gets no CORS headers, even when an allowlist is
// configured.
func TestCORSNativeClientUnaffected(t *testing.T) {
ts := newCORSServer(t, "http://localhost:5173")
resp, err := http.Get(ts.URL + "/healthz") // no Origin header
if err != nil {
t.Fatalf("get: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("healthz status = %d, want 200", resp.StatusCode)
}
if got := resp.Header.Get("Access-Control-Allow-Origin"); got != "" {
t.Fatalf("Allow-Origin set for a no-Origin native client: %q", got)
}
}
+111 -8
View File
@@ -1,8 +1,10 @@
package membership package membership
import ( import (
"fmt"
"net" "net"
"net/http" "net/http"
"strings"
"sync" "sync"
"time" "time"
@@ -78,16 +80,117 @@ func (l *ipRateLimiter) reapLocked(now time.Time) {
} }
} }
// clientIP extracts the source IP of an HTTP request, stripping the port. It // clientIP extracts the rate-limit key for a request: the source IP, with the
// trusts the transport's RemoteAddr only (no X-Forwarded-For parsing): a public // port stripped. By default it trusts the transport's RemoteAddr ONLY (no
// deployment terminates TLS at this process or behind a proxy that the operator // X-Forwarded-For parsing): honoring an attacker-supplied header would let a
// controls, and honoring an attacker-supplied header would let a single IP fan // single IP fan its quota across forged identities. When the operator runs the
// its quota across forged identities. If parsing fails the whole RemoteAddr is // control plane behind a reverse proxy they control (the same-origin Caddy
// used as the key (still a stable per-connection bucket). // deployment), SetTrustedProxies names that proxy's address(es); only then, and
func clientIP(r *http.Request) string { // only when the immediate peer is one of them, is the forwarded client IP
// believed. This keeps the per-IP rate limit meaningful behind the proxy, where
// every request would otherwise share the proxy's single IP. If parsing fails the
// whole RemoteAddr is used as the key (still a stable per-connection bucket).
func (s *Server) clientIP(r *http.Request) string {
host, _, err := net.SplitHostPort(r.RemoteAddr) host, _, err := net.SplitHostPort(r.RemoteAddr)
if err != nil { if err != nil {
return r.RemoteAddr host = r.RemoteAddr
}
if !s.trustedProxies.has(host) {
return host
}
if fwd := forwardedClientIP(r, s.trustedProxies); fwd != "" {
return fwd
} }
return host return host
} }
// forwardedClientIP returns the real client IP a trusted proxy reported, or "" if
// none is present. X-Forwarded-For is read RIGHT-TO-LEFT: the rightmost entry is
// the one our immediate (trusted) proxy appended and therefore cannot be spoofed
// by the client, which can only prepend entries to the left. Trusted-proxy hops
// are skipped so a chain of proxies we own resolves to the first address none of
// them owns — the actual external client. X-Real-IP is a single-value fallback for
// proxies that set it instead. A non-trusted immediate peer never reaches here, so
// a direct attacker's forged header is ignored entirely.
func forwardedClientIP(r *http.Request, trusted trustedProxyMatcher) string {
if xff := r.Header.Get("X-Forwarded-For"); xff != "" {
parts := strings.Split(xff, ",")
for i := len(parts) - 1; i >= 0; i-- {
ip := strings.TrimSpace(parts[i])
if ip == "" || trusted.has(ip) {
continue
}
if net.ParseIP(ip) != nil {
return ip
}
}
}
if xrip := strings.TrimSpace(r.Header.Get("X-Real-IP")); xrip != "" {
if net.ParseIP(xrip) != nil {
return xrip
}
}
return ""
}
// trustedProxyMatcher is the set of reverse-proxy addresses whose forwarding
// headers may be honored. The zero value (nil) matches nothing, so the default
// behavior is RemoteAddr-only.
type trustedProxyMatcher []*net.IPNet
// SetTrustedProxies configures the proxies whose X-Forwarded-For / X-Real-IP this
// server trusts for the per-IP rate limit. Each entry is an IP (treated as a /32
// or /128) or a CIDR. It returns an error on the first unparseable entry and
// leaves the previous configuration unchanged. Passing no entries clears the set.
func (s *Server) SetTrustedProxies(entries []string) error {
m, err := parseTrustedProxies(entries)
if err != nil {
return err
}
s.trustedProxies = m
return nil
}
// parseTrustedProxies turns a list of IPs/CIDRs into a matcher. A bare IP becomes
// a host route (/32 for IPv4, /128 for IPv6); blanks are skipped.
func parseTrustedProxies(entries []string) (trustedProxyMatcher, error) {
var m trustedProxyMatcher
for _, e := range entries {
e = strings.TrimSpace(e)
if e == "" {
continue
}
if _, ipnet, err := net.ParseCIDR(e); err == nil {
m = append(m, ipnet)
continue
}
ip := net.ParseIP(e)
if ip == nil {
return nil, fmt.Errorf("trusted proxy %q is not an IP or CIDR", e)
}
bits := 32
if ip.To4() == nil {
bits = 128
}
m = append(m, &net.IPNet{IP: ip, Mask: net.CIDRMask(bits, bits)})
}
return m, nil
}
// has reports whether host (an IP string with no port) falls inside any trusted
// range. A nil matcher and an unparseable host both report false.
func (m trustedProxyMatcher) has(host string) bool {
if len(m) == 0 {
return false
}
ip := net.ParseIP(host)
if ip == nil {
return false
}
for _, n := range m {
if n.Contains(ip) {
return true
}
}
return false
}
+113
View File
@@ -0,0 +1,113 @@
package membership
import (
"net/http"
"testing"
)
// TestClientIPTrustedProxy covers the rate-limit key extraction behind a reverse
// proxy: forwarding headers are believed ONLY when the immediate peer is a
// configured trusted proxy, and never otherwise. This is what keeps the per-IP
// rate limit per-client once the control plane runs behind the same-origin Caddy
// proxy, without opening a quota-fanning hole for a direct attacker.
func TestClientIPTrustedProxy(t *testing.T) {
const caddy = "135.125.201.30"
cases := []struct {
name string
proxies []string
remote string
xff string
xRealIP string
want string
}{
{
name: "no trusted proxies ignores XFF",
remote: "203.0.113.7:5000",
xff: "1.2.3.4",
want: "203.0.113.7",
},
{
name: "trusted proxy honors XFF client",
proxies: []string{caddy},
remote: caddy + ":4451",
xff: "198.51.100.23",
want: "198.51.100.23",
},
{
name: "loopback proxy honors XFF (magnus-local hop)",
proxies: []string{"127.0.0.1/32", "::1/128"},
remote: "127.0.0.1:33344",
xff: "198.51.100.99",
want: "198.51.100.99",
},
{
name: "untrusted peer cannot spoof XFF",
proxies: []string{caddy},
remote: "203.0.113.7:5000",
xff: "10.0.0.1",
want: "203.0.113.7",
},
{
name: "XFF read right-to-left, trusted hops skipped",
proxies: []string{caddy},
remote: caddy + ":4451",
xff: "198.51.100.23, " + caddy,
want: "198.51.100.23",
},
{
name: "client-prepended forgery is skipped, real appended wins",
proxies: []string{caddy},
remote: caddy + ":4451",
xff: "9.9.9.9, 198.51.100.23",
want: "198.51.100.23",
},
{
name: "X-Real-IP fallback when no XFF",
proxies: []string{caddy},
remote: caddy + ":4451",
xRealIP: "198.51.100.77",
want: "198.51.100.77",
},
{
name: "trusted peer but no forwarding header falls back to peer",
proxies: []string{caddy},
remote: caddy + ":4451",
want: caddy,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
s := &Server{}
if len(tc.proxies) > 0 {
if err := s.SetTrustedProxies(tc.proxies); err != nil {
t.Fatalf("SetTrustedProxies(%v): %v", tc.proxies, err)
}
}
r, _ := http.NewRequest(http.MethodGet, "/rooms", nil)
r.RemoteAddr = tc.remote
if tc.xff != "" {
r.Header.Set("X-Forwarded-For", tc.xff)
}
if tc.xRealIP != "" {
r.Header.Set("X-Real-IP", tc.xRealIP)
}
if got := s.clientIP(r); got != tc.want {
t.Fatalf("clientIP = %q, want %q", got, tc.want)
}
})
}
}
// TestParseTrustedProxiesRejectsGarbage proves a malformed entry is a hard error
// (the command turns it into a startup failure) rather than a silently ignored
// misconfiguration that would leave the rate limit collapsed behind the proxy.
func TestParseTrustedProxiesRejectsGarbage(t *testing.T) {
if _, err := parseTrustedProxies([]string{"not-an-ip"}); err == nil {
t.Fatal("expected error for non-IP/CIDR entry, got nil")
}
if _, err := parseTrustedProxies([]string{"10.0.0.0/8", "127.0.0.1"}); err != nil {
t.Fatalf("valid entries rejected: %v", err)
}
}
+82 -1
View File
@@ -87,6 +87,27 @@ type Server struct {
// posture a secure cluster requires (audit 0008 N1). It is set by the command; // posture a secure cluster requires (audit 0008 N1). It is set by the command;
// the zero value (all false) reflects an unsecured dev node. // the zero value (all false) reflects an unsecured dev node.
Posture Posture Posture Posture
// AllowedOrigins is the CORS allowlist of browser Origin headers permitted to
// call the control plane cross-origin. It exists so a browser-native client
// (uniweb) can talk to membershipd directly, the way the Go/Kotlin clients
// already do over a non-browser transport (issue uniweb/0001). Native clients
// send no Origin header and are unaffected. The zero value (empty) keeps CORS
// OFF — no Access-Control headers are emitted and the server behaves exactly as
// before — so this is opt-in per deployment. Entries are matched exactly (scheme
// + host + port); never use "*" with credentials. Set by the command from a flag.
AllowedOrigins []string
// trustedProxies names the reverse proxies whose forwarding headers
// (X-Forwarded-For / X-Real-IP) the rate limiter is allowed to believe. It
// exists for the same-origin deployment where a single proxy (Caddy) fronts
// the control plane: without it every proxied request would share the proxy's
// one IP and collapse the per-IP rate limit into a single bucket for the whole
// world. Only when the immediate peer is one of these addresses is the
// forwarded client IP trusted; the zero value (nil) trusts nobody, preserving
// the RemoteAddr-only behavior that predates the flag. Set by the command via
// SetTrustedProxies. See clientIP.
trustedProxies trustedProxyMatcher
} }
// Posture describes the security posture a membershipd node runs with. It is // Posture describes the security posture a membershipd node runs with. It is
@@ -143,10 +164,19 @@ func (s *Server) UseReplicatedNonces(js jetstream.JetStream, replicas int) error
func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
now := time.Now() now := time.Now()
// CORS runs before everything else so a browser preflight never pays the rate
// limit or auth cost. When the request carries an allowed Origin we echo the
// Access-Control headers; a preflight (OPTIONS) is answered here and short-
// circuits the pipeline. With an empty allowlist this is a no-op, so non-browser
// clients and untouched deployments behave exactly as before (issue uniweb/0001).
if s.applyCORS(w, r) {
return // preflight handled
}
// Per-IP rate limit runs first, ahead of auth and body reads, so a flood is // Per-IP rate limit runs first, ahead of auth and body reads, so a flood is
// shed at the cheapest possible point. The health probe is exempt so liveness // shed at the cheapest possible point. The health probe is exempt so liveness
// checks are never throttled. // checks are never throttled.
if !isAuthExempt(r) && !s.limiter.allow(clientIP(r), now) { if !isAuthExempt(r) && !s.limiter.allow(s.clientIP(r), now) {
writeErr(w, http.StatusTooManyRequests, "rate limit exceeded") writeErr(w, http.StatusTooManyRequests, "rate limit exceeded")
return return
} }
@@ -221,6 +251,57 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
s.mux.ServeHTTP(w, r.WithContext(withSigner(r.Context(), res.endpoint, res.pubHex))) s.mux.ServeHTTP(w, r.WithContext(withSigner(r.Context(), res.endpoint, res.pubHex)))
} }
// applyCORS handles cross-origin requests for the control plane. When the request
// carries an Origin in the allowlist it sets the Access-Control-Allow-* response
// headers so the browser accepts the eventual response; when the request is a CORS
// preflight (OPTIONS) it writes the preflight reply and returns true so ServeHTTP
// short-circuits before the rate limiter and auth ever run. It returns false for
// every non-preflight request — including same-origin and native clients that send
// no Origin header — leaving the normal pipeline to run unchanged. With an empty
// AllowedOrigins it never sets a header (CORS is off): the opt-in default.
func (s *Server) applyCORS(w http.ResponseWriter, r *http.Request) (preflight bool) {
origin := r.Header.Get("Origin")
allowed := origin != "" && s.originAllowed(origin)
if allowed {
h := w.Header()
h.Set("Access-Control-Allow-Origin", origin)
// Vary: Origin so a cache never serves an allow-listed response to another
// origin. Add (not Set) to preserve any Vary the handler may add later.
h.Add("Vary", "Origin")
h.Set("Access-Control-Allow-Methods", "GET, POST, OPTIONS")
// Allow the control-plane request-auth headers a browser client signs every
// request with (busauth.signedHeaders), or the browser's CORS preflight blocks
// the real request. Content-Type/Authorization stay for JSON bodies.
h.Set("Access-Control-Allow-Headers", "Content-Type, Authorization, X-Unibus-Pub, X-Unibus-Ts, X-Unibus-Nonce, X-Unibus-Sig")
h.Set("Access-Control-Max-Age", "600")
}
if r.Method == http.MethodOptions {
// Answer the preflight here so it never reaches the rate limiter or auth. An
// allowed origin gets 204 with the headers above; a disallowed or missing
// origin gets 403 with no Access-Control headers, so the browser blocks the
// real cross-origin request.
if allowed {
w.WriteHeader(http.StatusNoContent)
} else {
w.WriteHeader(http.StatusForbidden)
}
return true
}
return false
}
// originAllowed reports whether origin is in the CORS allowlist. Matching is exact
// (scheme + host + port): a browser Origin is an opaque string, so an exact compare
// is both correct and the safest policy (no wildcard, no suffix tricks).
func (s *Server) originAllowed(origin string) bool {
for _, o := range s.AllowedOrigins {
if o == origin {
return true
}
}
return false
}
// isBodyTooLarge reports whether err is the sentinel returned by MaxBytesReader // isBodyTooLarge reports whether err is the sentinel returned by MaxBytesReader
// when the body exceeds its limit, so the middleware can map it to 413. // when the body exceeds its limit, so the middleware can map it to 413.
func isBodyTooLarge(err error) bool { func isBodyTooLarge(err error) bool {