fix(0005c): bound aggregate buffered memory with a global in-flight byte limiter
The H1 fix bounds each request (1 MiB control / 16 MiB blob) and the per-IP rate
limiter throttles a single source, but neither bounds the AGGREGATE memory across
concurrent requests. The re-audit (report 0006, N2) drove RSS to ~1.42 GB with 40
concurrent 16 MiB uploads, and noted that a multi-IP (botnet) flood scales without
a ceiling because the rate limit is per-IP.
Fix: a global, non-blocking, byte-counting limiter (pkg/membership/inflight.go).
ServeHTTP reserves a POST's worst-case buffered size (its route ceiling) from the
limiter before reading the body, and releases it when the request finishes. When
the global cap (maxInflightBytes = 128 MiB) is reached, further POSTs are shed
with 503 (backpressure) rather than parking goroutines, so total bytes buffered
in flight stays bounded regardless of connection count or source-IP spread. GETs
carry no body and do not consume the budget.
The limiter is implemented inside unibus (not delegated to the fn-registry, where
a generic concurrency primitive would normally live) because functions/core pulls
transitive deps requiring CGO (mattn/go-sqlite3) and external modules that are
incompatible with unibus's CGO_ENABLED=0 build, and because this work is scoped
to the unibus sub-repo. The type/method comments document this.
Verification:
- pkg/membership/inflight_test.go: TestInflightLimiter{Basics,Disabled,Concurrent}
cover golden/edge/error/disabled/over-release and a -race concurrency invariant
(inFlight returns to 0, never exceeds cap).
- pkg/membership/dos_concurrency_test.go: TestReaudit_DoSConcurrency fires 40
concurrent 16 MiB uploads from distinct IPs (the multi-IP shape) against a 48 MiB
test cap -> 200=3 503=37, RSS delta ~93 MiB (bound 256 MiB), inFlight()==0, and a
fresh upload still 200. With the limiter disabled the test fails (200=40 503=0),
confirming it is a real regression guard.
- CGO_ENABLED=0 go build ./... && go vet ./... && go test -count=1 ./... green;
CGO_ENABLED=1 go test -race ./pkg/membership/ green.
Residual (documented): under enforce the body is buffered twice (auth verify +
handler), so real RSS is ~2x the reserved bytes; closing that fully means
streaming blobs to disk (overlaps H9 / issue 0002).
Refs: report 0006 N2, issue 0005c.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,97 @@
|
||||
package membership
|
||||
|
||||
import (
|
||||
"sync"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestInflightLimiterBasics covers the limiter contract: granting within the cap
|
||||
// (golden), the exact boundary (edge), refusal over the cap without mutating the
|
||||
// counter (error), the disabled mode, and the defensive clamp on over-release.
|
||||
func TestInflightLimiterBasics(t *testing.T) {
|
||||
l := newInflightLimiter(100)
|
||||
|
||||
// Golden: a reservation within the cap is granted and reflected.
|
||||
if !l.tryAcquire(60) {
|
||||
t.Fatalf("acquire 60 within cap 100 should grant")
|
||||
}
|
||||
if l.inFlight() != 60 {
|
||||
t.Fatalf("inFlight = %d, want 60", l.inFlight())
|
||||
}
|
||||
|
||||
// Edge: exactly reaching the cap (60+40 == 100) is granted.
|
||||
if !l.tryAcquire(40) {
|
||||
t.Fatalf("acquire to the exact cap should grant")
|
||||
}
|
||||
if l.inFlight() != 100 {
|
||||
t.Fatalf("inFlight = %d, want 100", l.inFlight())
|
||||
}
|
||||
|
||||
// Error: one more byte over the full cap is refused, and the counter is left
|
||||
// untouched (a refused reservation reserves nothing).
|
||||
if l.tryAcquire(1) {
|
||||
t.Fatalf("acquire over a full cap must be refused")
|
||||
}
|
||||
if l.inFlight() != 100 {
|
||||
t.Fatalf("a refused acquire must not change inFlight; got %d", l.inFlight())
|
||||
}
|
||||
|
||||
// Release frees capacity again.
|
||||
l.release(100)
|
||||
if l.inFlight() != 0 {
|
||||
t.Fatalf("inFlight after full release = %d, want 0", l.inFlight())
|
||||
}
|
||||
|
||||
// Defensive: an over-release never drives the counter negative.
|
||||
l.release(50)
|
||||
if l.inFlight() != 0 {
|
||||
t.Fatalf("over-release must clamp at 0; got %d", l.inFlight())
|
||||
}
|
||||
}
|
||||
|
||||
// TestInflightLimiterDisabled verifies that a non-positive cap disables the
|
||||
// limiter: every reservation is granted and nothing is tracked (the loopback/dev
|
||||
// posture).
|
||||
func TestInflightLimiterDisabled(t *testing.T) {
|
||||
for _, max := range []int64{0, -1} {
|
||||
l := newInflightLimiter(max)
|
||||
if !l.tryAcquire(1 << 30) {
|
||||
t.Fatalf("disabled limiter (max=%d) must always grant", max)
|
||||
}
|
||||
if l.inFlight() != 0 {
|
||||
t.Fatalf("disabled limiter must not track usage; got %d", l.inFlight())
|
||||
}
|
||||
l.release(1 << 30) // no-op, must not panic
|
||||
}
|
||||
}
|
||||
|
||||
// TestInflightLimiterConcurrent hammers the limiter from many goroutines with
|
||||
// equal-sized acquire/release pairs and asserts the invariant never breaks: the
|
||||
// counter returns to 0 and never exceeds the cap. Run with -race for the memory
|
||||
// model guarantee.
|
||||
func TestInflightLimiterConcurrent(t *testing.T) {
|
||||
const cap = 1000
|
||||
const chunk = 7
|
||||
l := newInflightLimiter(cap)
|
||||
|
||||
var wg sync.WaitGroup
|
||||
for g := 0; g < 64; g++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
for i := 0; i < 2000; i++ {
|
||||
if l.tryAcquire(chunk) {
|
||||
if f := l.inFlight(); f > cap {
|
||||
t.Errorf("inFlight %d exceeded cap %d", f, cap)
|
||||
return
|
||||
}
|
||||
l.release(chunk)
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
if l.inFlight() != 0 {
|
||||
t.Fatalf("after all goroutines, inFlight = %d, want 0", l.inFlight())
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user