fix(0005c): bound aggregate buffered memory with a global in-flight byte limiter
The H1 fix bounds each request (1 MiB control / 16 MiB blob) and the per-IP rate
limiter throttles a single source, but neither bounds the AGGREGATE memory across
concurrent requests. The re-audit (report 0006, N2) drove RSS to ~1.42 GB with 40
concurrent 16 MiB uploads, and noted that a multi-IP (botnet) flood scales without
a ceiling because the rate limit is per-IP.
Fix: a global, non-blocking, byte-counting limiter (pkg/membership/inflight.go).
ServeHTTP reserves a POST's worst-case buffered size (its route ceiling) from the
limiter before reading the body, and releases it when the request finishes. When
the global cap (maxInflightBytes = 128 MiB) is reached, further POSTs are shed
with 503 (backpressure) rather than parking goroutines, so total bytes buffered
in flight stays bounded regardless of connection count or source-IP spread. GETs
carry no body and do not consume the budget.
The limiter is implemented inside unibus (not delegated to the fn-registry, where
a generic concurrency primitive would normally live) because functions/core pulls
transitive deps requiring CGO (mattn/go-sqlite3) and external modules that are
incompatible with unibus's CGO_ENABLED=0 build, and because this work is scoped
to the unibus sub-repo. The type/method comments document this.
Verification:
- pkg/membership/inflight_test.go: TestInflightLimiter{Basics,Disabled,Concurrent}
cover golden/edge/error/disabled/over-release and a -race concurrency invariant
(inFlight returns to 0, never exceeds cap).
- pkg/membership/dos_concurrency_test.go: TestReaudit_DoSConcurrency fires 40
concurrent 16 MiB uploads from distinct IPs (the multi-IP shape) against a 48 MiB
test cap -> 200=3 503=37, RSS delta ~93 MiB (bound 256 MiB), inFlight()==0, and a
fresh upload still 200. With the limiter disabled the test fails (200=40 503=0),
confirming it is a real regression guard.
- CGO_ENABLED=0 go build ./... && go vet ./... && go test -count=1 ./... green;
CGO_ENABLED=1 go test -race ./pkg/membership/ green.
Residual (documented): under enforce the body is buffered twice (auth verify +
handler), so real RSS is ~2x the reserved bytes; closing that fully means
streaming blobs to disk (overlaps H9 / issue 0002).
Refs: report 0006 N2, issue 0005c.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -35,6 +35,14 @@ const (
|
||||
// MaxHeaderBytes caps request header size; wired into the http.Server by the
|
||||
// command. Exported so the bound lives next to its body-size siblings.
|
||||
MaxHeaderBytes = 1 << 20 // 1 MiB
|
||||
// maxInflightBytes is the GLOBAL cap on request-body bytes buffered across all
|
||||
// concurrent requests (audit N2). The per-request ceilings above bound one
|
||||
// request; this bounds the sum, so a concurrent (even multi-IP) flood of
|
||||
// max-size uploads cannot drive the resident set without limit. 128 MiB allows
|
||||
// ~8 concurrent 16 MiB blob uploads or ~128 concurrent control requests before
|
||||
// further POSTs are shed with 503 — generous for an interactive bus, bounded
|
||||
// for an attacker.
|
||||
maxInflightBytes = 128 << 20 // 128 MiB
|
||||
)
|
||||
|
||||
// Per-IP rate-limit defaults for the control plane. Tuned for an interactive
|
||||
@@ -62,6 +70,7 @@ type Server struct {
|
||||
authMode AuthMode
|
||||
nonces nonceStore
|
||||
limiter *ipRateLimiter
|
||||
inflight *inflightLimiter
|
||||
|
||||
// RequireEncryptedRooms, when true, refuses to create cleartext (ModeNATS)
|
||||
// rooms. It is the minimum-defensive control for the data plane (audit H4):
|
||||
@@ -87,6 +96,7 @@ func NewServer(store Store, blobs blobstore.Store, authMode AuthMode) *Server {
|
||||
authMode: authMode,
|
||||
nonces: newMemNonceCache(nonceTTL, maxNonceCacheEntries),
|
||||
limiter: newIPRateLimiter(defaultRatePerSec, defaultRateBurst, rateBucketTTL),
|
||||
inflight: newInflightLimiter(maxInflightBytes),
|
||||
}
|
||||
s.routes()
|
||||
return s
|
||||
@@ -139,6 +149,22 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
r.Body = http.MaxBytesReader(w, r.Body, limit)
|
||||
|
||||
// Aggregate memory bound (audit N2): the per-request ceiling above and the
|
||||
// per-IP rate limit do not cap the TOTAL bytes buffered across concurrent
|
||||
// requests. A POST reserves its worst-case buffered size (its route ceiling)
|
||||
// from a global limiter before the body is read, and is shed with 503 when the
|
||||
// cap is reached, so the resident set stays bounded under a concurrent (even
|
||||
// multi-IP) upload flood instead of growing linearly with the number of
|
||||
// connections. Reservation is released when the request finishes. Only POSTs
|
||||
// buffer a body; GETs carry none, so they do not consume the budget.
|
||||
if r.Method == http.MethodPost {
|
||||
if !s.inflight.tryAcquire(limit) {
|
||||
writeErr(w, http.StatusServiceUnavailable, "server busy: too many concurrent uploads in flight")
|
||||
return
|
||||
}
|
||||
defer s.inflight.release(limit)
|
||||
}
|
||||
|
||||
if s.authMode == AuthOff || isAuthExempt(r) {
|
||||
s.mux.ServeHTTP(w, r)
|
||||
return
|
||||
|
||||
Reference in New Issue
Block a user