HTTP server qualification

This page records what was checked before std.http.server was labelled production-ready, what that showed, and how to deploy it. The server is production-ready: every gate passes; see Gates. HTTP/2 and HTTP/3 need separate adoption decisions and are not part of these gates.

The suites are in tests/test_std_http_server_qualification.rs, with the request corpus in tests/fixtures/http_server/, and in tests/test_std_http_server_proxy.rs, which runs the server behind nginx and Caddy in Docker (see Reverse proxies). The benchmark is examples/http_server_bench.py. Everything runs on loopback; only the proxy tests need a network, once, to pull their images.

The current side-by-side TLS comparison records worker and actor throughput, handshake latency, idle RSS and trap recovery after actor TLS heap isolation and completion-credit changes. The original baselines on this page remain historical measurements of their recorded revision.

Gates

Supported platforms

x86-64 Linux with the GNU C library, the only target of std.net and std.ready (epoll). Results here were taken on Linux 7.2 (Fedora 44). No other target is supported.

Application error composition

The installed offline package test also checks B14/B71 through its shared CSV summary dependency. The CLI reports invalid integer data with its record number and ParseError, malformed row width with the complete CsvError, and a missing file with its nested stream/I/O cause. Each report retains the operation and path, prints no partial JSON, and exits 1; argument errors still exit 2 and help exits 0.

The JSON service returns those same causes as Context[SummaryError] in its handler error sum and reports them before propagation. One client repeats all three failures and a successful summary three times. The test checks identical CLI/service diagnostics, generic 500 responses without private error details, successful requests after failures, nine handler errors among twelve requests, and no protocol errors. After the client, workers, CSV readers and listener have dropped, the process has the same number of open descriptors as before the run. This qualifies application composition over the CSV/HTTP APIs. The client now uses typed head parsing, writes and fixed/chunked/close-delimited framing. Separate typed-stream and client checks cover exact partial progress, original transport causes, interruption, bounded heads, trailers, compatibility readers and connection recovery; HTTPS checks cover verified and tunnelled requests.

Handler errors and traps

A handler reports a problem by returning Err; the server answers it, counts it and goes on serving (see When a handler fails). A fatal runtime error is different. An index out of range, integer overflow, division by zero, unwrap of an Err, or worker stack exhaustion terminates the whole process at once, without unwinding, with a message such as error: main.plenty:3:11: index out of bounds: index 7, length 3 and exit status 1. Every connection is cut off and no Report is returned.

So in a handler:

• Check indexes and arithmetic on anything that comes from a request, use optional lookups and ?, and return Err rather than calling unwrap.
• Run the server under a supervisor that restarts it (systemd Restart=on-failure, a container restart policy).
• Allocation failure is not a trap: it closes the affected connection.

Baselines

examples/http_server_bench.py builds examples/http_server_bench.plenty with --release, runs it with 4 workers on CPUs 0, 2, 4 and 6 and ApacheBench 2.3 on CPUs 8, 10, 12 and 14 (one hardware thread per physical core, so at most ten threads are busy), and repeats each workload against a fresh server. With --tls it builds examples/https_server_bench.plenty instead, the same handler served by std.http.tls with the ECDSA P-256 certificate in tests/fixtures/tls.

Measured on 2026-10-18 at revision 32c48c0, built by rustc 1.99.0 (target x86_64-unknown-linux-gnu (CPU baseline: x86-64/SSE2)), on an AMD Ryzen 7 7840HS w/ Radeon 780M Graphics with Linux 7.2.9-201.fc44.x86_64, three runs per workload. Other builds were running (load average about 4.5 at the start of each run), so treat the ranges as noise bounds.

HTTPS

The same workloads over TLS 1.3 (TLS_AES_256_GCM_SHA384, X25519, ECDSA P-256 server key; rustls 0.23.46 with ring), measured in the same session:

Each new HTTPS connection is a full handshake: the server keeps no session cache and sends no tickets, so clients cannot resume. ApacheBench cannot measure that workload: it counts connections closed without close_notify as failed. The script runs curl 8.18.0 (OpenSSL 3.5.9) instead: four processes on the client CPUs, each with --parallel-max 4, TCP_NODELAY (curl’s default) and no certificate verification. The same curl method against the plain server gave 21397 (21225–22032) requests/s, p50 / p99 0.67 / 1.05 ms, so the handshake, not the client, sets the HTTPS rate.

With 1000 parked keep-alive connections the server’s resident memory grew from 4364 KiB to 6680 KiB over plain HTTP, about 2.3 KiB per idle connection, and from 5928 KiB to 48552 KiB over HTTPS, about 42.6 KiB per idle connection: each TLS session keeps its rustls buffers and keys. Size idle_connections for HTTPS with that in mind.

Latency is per request as the client sees it. Peak RSS is the server’s VmHWM. 16k is a 16 KiB response; the POST echoes a 1 KiB body. Each keep-alive connection closes after max_requests (1000) requests, so ApacheBench reconnects. An earlier plain run, on 2026-10-17 at load average about 8, gave 9239 to 16744 requests/s for the same workloads; the difference is machine load, not a server change.

Limits of these numbers: one machine, loopback only, a client that is itself a bottleneck at high rates, small fixed responses, one certificate type and no TLS session resumption. They are a regression baseline, not a capacity plan.

Before this qualification, a response whose head and body went out as two writes (any body over the 4 KiB write buffer) waited about 40 ms for a delayed acknowledgement: the 16 KiB workload ran at 388 requests/s. The server now turns off Nagle’s algorithm on accepted connections.

Until issue #148 was fixed, a client that leaves Nagle’s algorithm on (ApacheBench, Python’s socket) waited about 40 ms on each new HTTPS connection: a TLS 1.3 server sends nothing after the client’s Finished, so the client held its first request until the kernel’s delayed acknowledgement. ab -n 20 -c 1 without -k took 42–44 ms per request. The server now acknowledges the Finished at once; on 2026-10-18 the same client took 0.76 ms per request over 200 requests (unpinned, not the baseline method above).

Slow request heads

Until issue #132 was fixed, a worker read each request head as it arrived, so a client that sent part of a head held a worker until head_timeout. Now a partial head, like a partial TLS handshake, waits on the accepting thread (see Idle connections). Measured on 2026-10-17 with the scenario of slow_heads_do_not_hold_workers below, also run with the defaults: clients that each send GET / HTTP/1.1\r\nHost: x and stall, then one complete request.

The slow clients still count against idle_connections (default 256): with that many parked, new plaintext connections get 503 until slots become available (new HTTPS connections close). Already-admitted partial heads and handshakes returned by workers can park beyond that admission cap; see the bounded overflow and parking-disabled behavior. With 1000 parked partial heads the server’s resident memory grew from 4548 KiB to 6396 KiB, about 1.9 KiB per connection. Size idle_connections and the descriptor limit for the slow clients an exposed server must outlast. Keep-alive and new-connection throughput showed no change beyond run-to-run noise on a loaded machine (load average about 8). slow_heads_do_not_hold_workers in the qualification suite and partial_heads_wait_without_a_worker and https_partial_handshakes_and_heads_wait_without_a_worker in test_std_http_server.rs check this.

Refusals and resets

Until issue #138 was fixed, a refusal of a request the server had not read to the end closed the socket with request bytes unread, so Linux reset the connection. Measured on 2026-10-17 against examples/http_server_bench.plenty, ten connections per case, each sending the request and then reading to the end:

The results were the same with IDLE=0, where the worker lingers. Before the fix a client reading to the end saw ECONNRESET, sometimes after reading the answer. Now the server ends its output, discards up to 16 MiB (64 KiB until the proxy tests below) for up to 2 s, then closes (see Lingering close). refusals_end_cleanly_instead_of_resetting and tls_refusals_end_with_close_notify in the qualification suite check this.

Fuzzing

fuzz/http_server/main.plenty is a libFuzzer target: plenty --fuzz counts every block the compiled program runs, including the standard library’s, and libFuzzer keeps the inputs that reach new ones. Each input is one client connection served by serve_connection over an in-memory stream. Its first byte selects the read size (1 to 64 bytes), small limits (a 64-byte line, a 256-byte head, 4 fields, a 16-byte body) and a limit of two requests per connection; the rest is what the client sends. The routes are those of the corpus. Response bodies never repeat the request, so the target finds every response head in what the server wrote and traps on a status outside 100, 200, 400, 404, 405, 413, 431, 500, 501 and 505. Traps of the server itself, stack exhaustion and hangs over 10 s are findings too.

fuzz/http_server/run.py builds the target with --release, seeds it with every corpus case in four variants (64-byte reads, 7-byte reads, 1-byte reads, 7-byte reads with small limits) and runs it with the HTTP dictionary fuzz/http_server/http.dict. Findings go to WORK/findings; the default WORK is target/fuzz-http-server. Findings persist across runs: the script lists only the files the current run added, and exits nonzero only for those or for a libFuzzer failure.

cargo build --release --locked
python3 fuzz/http_server/run.py --time 1800 --jobs 2
target/fuzz-http-server/http_server_fuzzer FINDING    # replay one finding

On 2026-10-18, on the machine of the baselines, two jobs ran for 30 minutes each (rustc 1.99.0, clang 22.1.8’s libFuzzer), starting from the corpus of an earlier 10-minute run: 16.9 million inputs, about 4,700 per second per job, with no finding. 3,777 of the program’s 13,878 blocks were reached, nearly all within the first seconds; most of the rest belong to functions the target never calls, such as serve and TLS, which a fuzz build keeps.

To act on a finding, replay it, reduce it with -minimize_crash=1 -runs=100000, and handle it as a parser defect under Security fixes: its request joins the corpus, and so the seeds.

Not covered: TLS records (rustls and ring are fuzzed upstream), the std.ready parking of real sockets, and timeouts, since the in-memory stream never waits.

Reverse proxies

tests/test_std_http_server_proxy.rs puts nginx and Caddy, each in its stock image, in front of the server, and drives them with curl and raw sockets. The server runs on the host (it links the host’s C library, so it does not run on Alpine); the proxy container uses --network host and reaches it on loopback. The configurations are tests/fixtures/proxy/nginx.conf and tests/fixtures/proxy/Caddyfile; the server is tests/fixtures/proxy/server.plenty.

Each test runs against both proxies. Clients connect from 127.0.0.2, so the handler can tell them from the proxy at 127.0.0.1.

The harness reserves distinct TCP ports and the TLS port’s UDP socket through configuration setup, then releases them immediately before starting Docker. Caddy also opens that UDP port for HTTP/3; this is not HTTP/3 qualification. Process-local logical leases retain both port numbers across that handoff and until the proxy’s Drop cleanup completes, including startup failure and unwind. Concurrent cooperating fixtures reject an already claimed port in either role and select another pair; only registry operations hold the mutex, not Docker startup or scenarios. Deterministic non-Docker tests force reuse during the socket-free gap, check cross-role exclusion and RAII release, and stress eight concurrent reservers (issue #719). These are helper tests, not renewed real-proxy qualification. A separate exact batch on the reviewed port-ownership source passes all eight worker and two actor nginx/Caddy cases with two test threads (local PR #918), including actual container removal and unchanged foreign container state. This is bounded proxy qualification, not a whole-workspace or performance result. Unrelated processes remain outside this ownership registry. Host networking still has a socket handoff window, so readiness checks a unique proxy-local response over both HTTP and verified TLS, without adding upstream requests. An unexpected HTTP status or identity fails immediately rather than being retried. Shutdown setup failures retain response headers and body, listener/upstream ports, container logs and upstream stderr. These checks harden endpoint identification; they do not establish the cause of the intermittent pre-shutdown 400 reported in issue #647.

Differences between the proxies, none of them a server fault:

• nginx appends to a client’s X-Forwarded-For (203.0.113.9, 127.0.0.2); Caddy, trusting no proxies by default, replaces it (127.0.0.2). With only the proxy trusted, client_of gives 127.0.0.2 for both.
• nginx sends 100 Continue itself and buffers the body. Caddy streams it and also relays the server’s own 100 Continue, so the client sees two interim responses, which RFC 9110 section 15.2 requires clients to accept.
• Caddy offers HTTP/2 to TLS clients and speaks HTTP/1.1 to the server.
• After relaying an answer sent before a large body was read, Caddy closes the client’s connection with the rest of the body unread, so a client still sending may see a reset after the answer. Curl can stop on its upload error before reading the final response. The Caddy refusal and early-answer checks therefore receive concurrently with the upload and require the final status, a single correct Content-Length, and the exact complete body before accepting a reset. A lone 100 Continue, a truncated answer, or a timeout still fails. The nginx checks require curl to exit successfully. The other curl interoperability checks remain strict.
• nginx reuses connections to the server only with an upstream block with keepalive, proxy_http_version 1.1 and an empty Connection header, as in the test configuration.

The tests found one server defect (issue #142). A refusal, or an answer sent before the body was read, closed lingering after discarding at most 64 KiB. A proxy forwards the whole body regardless, so it was often still sending when the server closed; the connection reset under it, and it answered 502 instead of passing on the server’s answer. The budget is now 16 MiB, still within 2 s. Twenty requests per case, measured on 2026-10-17:

With 200 KiB bodies neither proxy saw a 502 under either budget.

Deployment

For a public service, put a reverse proxy in front of the server (see deploying behind a reverse proxy). The proxy also absorbs slow clients beyond what idle_connections bounds:

• Buffer whole request heads, and preferably bodies, before forwarding (nginx does both by default; keep proxy_request_buffering on). tests/fixtures/proxy/ has tested nginx and Caddy configurations.
• Connect to the server over loopback or a private network with proxy_http_version 1.1 and keep-alive, so connections are reused.
• Set the proxy’s client timeouts and connection limits per client; the server’s limits are per connection.

The server itself:

• Listen on loopback or a private address. Serve HTTPS directly only when no proxy is possible, and then keep head_timeout and handshake_timeout short and idle_connections above the number of slow clients expected.
• Raise the descriptor limit (LimitNOFILE in systemd) above idle_connections + queue + workers plus the program’s own files; when it runs out, accept fails, is counted in accept_errors and retried every 10 ms.
• Set events to a std.log channel sink and export the Report counts; rejected, timeouts and accept_errors show overload.
• Drain on SIGTERM with std.signal, and set the supervisor’s stop timeout (TimeoutStopSec in systemd) above drain_timeout.

Security fixes and backend updates

The HTTP modules are checked Plenty in the compiler’s embedded standard library; a fix reaches a program when it is rebuilt with a fixed compiler. For a defect in parsing, framing or limits:

1. Record it in the issue register with severity high, and add the triggering request to tests/fixtures/http_server/corpus.txt with its correct answer, so it stays fixed.
2. Fix it in stdlib/std/http.plenty or stdlib/std/http/, and run this suite with test_std_http.rs, test_std_http_server.rs, test_std_http_client.rs and test_std_http_helpers.rs.
3. Rebuild and redeploy every program that serves HTTP.

TLS uses vendored rustls and ring, pinned in Cargo.lock. Follow rustls and ring advisories and update them as described under TLS supply, then run test_std_tls.rs, test_std_http_https.rs and the TLS tests here. No other native backend is linked into a program that only serves HTTP.

Reproducing

cargo test --locked --all-features --test integration test_std_http_server_qualification::
PLENTY_DOCKER_TESTS=1 cargo test --locked --test integration test_std_http_server_proxy::
cargo build --release --locked
python3 examples/http_server_bench.py --compiler target/release/plenty --output target/http-bench --runs 3
python3 examples/http_server_bench.py --compiler target/release/plenty --output target/https-bench --runs 3 --tls
python3 fuzz/http_server/run.py --time 1800 --jobs 2

The curl tests are skipped when curl is not installed. The proxy tests need Docker and curl and are skipped, with a message, unless PLENTY_DOCKER_TESTS=1 (see reverse-proxy tests); the proxy_tests job in .github/workflows/ci.yml runs them on every push and pull request. The benchmark needs ApacheBench (ab) and taskset, and with --tls also curl. Fuzzing needs clang’s libFuzzer; without it the libFuzzer tests in test_fuzz.rs print a note and pass, unless PLENTY_FUZZ_TESTS=1. Busy machines distort the benchmark: check the load average first, and keep the total number of busy threads at or below the number of physical cores.