8b6a01d280
membershipd never called Server.UseReplicatedNonces, so every node kept a per-process anti-replay cache and a signed request accepted on node A could be replayed to node B (200+200). This wires the shared JetStream KV nonce bucket on any clustered node, closing the cross-node replay hole. Bootstrap: under enforce the service needs JetStream on its own embedded server, but the data plane only accepts allowlisted clients. Resolved with an ephemeral internal service identity the authenticator recognizes and grants full permissions (NewNkeyAuthenticatorACLInternal), connected over the in-process transport (no TLS/CA needed for the self-connection). Hard rule: --cluster-name != "" means the replicated nonce bucket is mandatory; if it cannot be created the node refuses to start (wireReplicatedNonces returns a fatal error) rather than run insecurely. Standalone nodes keep the in-memory cache unchanged (branch-by-abstraction: no JetStream dependency added). Changes: - busauth: NewNkeyAuthenticatorACLInternal + fullPermissions for the internal id. - cmd/membershipd: connectInternalJS (in-process, privileged) / connectExternalJS; wireReplicatedNonces helper; main wires it when clustered; --kv-replicas flag. Tests (regression of audit 0008 N3): - TestAttack0008_N3: 2 clustered nodes share the bucket, cross-node replay -> 401. - TestAttack0008_N3_StandaloneKeepsLocalCache: standalone needs no JetStream, same-node replay still 401. - TestAttack0008_N3_ClusteredRequiresJetStream: clustered + no JetStream -> fatal. - TestInternalConnPrivilegedUnderEnforce / ...OutsiderRejected: the privileged self-connection works under enforce and no other identity can claim it. CGO_ENABLED=0 go build/vet/test green; govulncheck 0 reachable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
85 lines
3.3 KiB
Go
85 lines
3.3 KiB
Go
package main
|
|
|
|
import (
|
|
"fmt"
|
|
|
|
cs "fn-registry/functions/cybersecurity"
|
|
|
|
"github.com/enmanuel/unibus/pkg/busauth"
|
|
"github.com/nats-io/nats.go"
|
|
"github.com/nats-io/nats.go/jetstream"
|
|
|
|
server "github.com/nats-io/nats-server/v2/server"
|
|
)
|
|
|
|
// connectInternalJS opens a privileged JetStream client from membershipd to its
|
|
// OWN embedded NATS server. This is the resolution of the "bootstrap cycle"
|
|
// (issue 0006a/c): the service needs JetStream to create the replicated nonce
|
|
// bucket and the control-plane KV, but under enforce the data plane only accepts
|
|
// allowlisted clients confined to their rooms. The connection therefore
|
|
// authenticates with the process's ephemeral internal identity — the identity the
|
|
// authenticator was built to recognize (NewNkeyAuthenticatorACLInternal) and
|
|
// grant full permissions — without ever appearing in the user allowlist.
|
|
//
|
|
// It uses the in-process transport (nats.InProcessServer), a Go pipe inside the
|
|
// process, so it bypasses TLS entirely: no CA wiring is needed for this
|
|
// self-connection even when the public data plane is TLS-only. useNkey mirrors
|
|
// whether the embedded server enforces auth: under enforce the internal identity
|
|
// presents its nkey; without enforce the server accepts an unauthenticated
|
|
// in-process client and the nkey is omitted.
|
|
//
|
|
// The caller owns the returned connection and must Close it on shutdown (after
|
|
// the JetStream context is no longer used).
|
|
func connectInternalJS(ns *server.Server, internalID cs.Identity, useNkey bool) (*nats.Conn, jetstream.JetStream, error) {
|
|
opts := []nats.Option{
|
|
nats.Name("membershipd-internal"),
|
|
nats.InProcessServer(ns),
|
|
}
|
|
if useNkey {
|
|
pub, sign, err := busauth.ClientNkey(internalID.SignPriv)
|
|
if err != nil {
|
|
return nil, nil, fmt.Errorf("internal nkey: %w", err)
|
|
}
|
|
opts = append(opts, nats.Nkey(pub, sign))
|
|
}
|
|
// The URL is ignored for an in-process connection; the InProcessServer option
|
|
// supplies the transport.
|
|
nc, err := nats.Connect("", opts...)
|
|
if err != nil {
|
|
return nil, nil, fmt.Errorf("connect internal nats: %w", err)
|
|
}
|
|
js, err := jetstream.New(nc)
|
|
if err != nil {
|
|
nc.Close()
|
|
return nil, nil, fmt.Errorf("internal jetstream: %w", err)
|
|
}
|
|
return nc, js, nil
|
|
}
|
|
|
|
// connectExternalJS opens a JetStream client to an EXTERNAL NATS the operator
|
|
// runs (membershipd started with --nats-url). Unlike the embedded path there is
|
|
// no in-process transport and no internal identity: the external server enforces
|
|
// its own auth, so membershipd connects as a plain client (optionally TLS-pinned
|
|
// to the bus CA). It is best-effort and intended for an operator-managed cluster;
|
|
// the standard unibus deploy uses the embedded server (connectInternalJS).
|
|
func connectExternalJS(natsURL, caPath string) (*nats.Conn, jetstream.JetStream, error) {
|
|
opts := []nats.Option{nats.Name("membershipd-internal")}
|
|
if caPath != "" {
|
|
tlsCfg, err := busauth.LoadCATLSConfig(caPath)
|
|
if err != nil {
|
|
return nil, nil, fmt.Errorf("load CA %q: %w", caPath, err)
|
|
}
|
|
opts = append(opts, nats.Secure(tlsCfg))
|
|
}
|
|
nc, err := nats.Connect(natsURL, opts...)
|
|
if err != nil {
|
|
return nil, nil, fmt.Errorf("connect external nats %q: %w", natsURL, err)
|
|
}
|
|
js, err := jetstream.New(nc)
|
|
if err != nil {
|
|
nc.Close()
|
|
return nil, nil, fmt.Errorf("external jetstream: %w", err)
|
|
}
|
|
return nc, js, nil
|
|
}
|