std.http.proxy

import std.http.proxy finds the client of a request that came through reverse proxies. A forwarding header is believed only from a proxy the program names. The module is checked Plenty in stdlib/std/http/proxy.plenty and adds no runtime primitives. This page also gives the deployment guidance for std.http.server.

import std.http.proxy
import std.net
from std.http import Header
from std.http.proxy import Forwarding, ProxyError

def main() -> Result[(), ProxyError | std.net.NetError | AllocError | IoError]:
    mut proxies = std.http.proxy.proxies(Forwarding.XForwardedFor)?
    proxies.trust("10.0.0.0/8")?
    headers = [Header("X-Forwarded-For", "198.51.100.7, 10.0.0.3"), Header("X-Forwarded-Proto", "https")]?
    for peer_text in ["10.0.0.2:443", "203.0.113.9:443"]?:
        peer = std.net.parse_address(&peer_text)?
        found = std.http.proxy.client(&peer, &headers, &proxies)?
        print(std.net.format_ip(found.address)?, found.scheme, found.proxied)?
    Ok(())

198.51.100.7 Option[str].Some("https") True
203.0.113.9 Option[str].Nothing False

Clients

The rule:

1. If the peer is not trusted, it is the client, and the forwarding headers are ignored.
2. Otherwise the entries of the chosen header are read from the right. Each trusted address is a proxy; the first untrusted one is the client.
3. If every entry is trusted, the leftmost is the client. If there are no entries, the peer is.

Entries left of the client were written by the client. Under X-Forwarded-For they are never parsed as addresses, but they still count toward max_hops. Under Forwarded every element is syntax-checked and counted, so a malformed element or too many elements anywhere in the header fails. The other header is always ignored, so a deployment that uses Forwarded cannot be fooled with X-Forwarded-For.

An entry is 192.0.2.1, 192.0.2.1:8080, 2001:db8::1 or [2001:db8::1]:8080; the port is dropped. An entry that must be read but is none of these, such as unknown or an obfuscated _name, fails with InvalidForwarded, as does a malformed Forwarded element. More than max_hops entries is TooManyHops. Empty comma-separated pieces are skipped. Quoted escapes must have a following byte; a trailing backslash is InvalidForwarded. Scanners stop at the end of a field without advancing an index beyond its length, including at the i64 representation limit. An IPv4-mapped IPv6 address, as a dual-stack listener reports IPv4 peers, is treated as the IPv4 address.

scheme is http or https, lowercased, when a trusted proxy reported it: the client’s element’s proto= for Forwarded, or the last X-Forwarded-Proto value, which the nearest proxy wrote. Any other scheme is InvalidForwarded. It is Nothing when the peer is not trusted.

trust refuses a network with bits set below its prefix length (10.0.0.1/8) with InvalidNetwork, so a typo cannot trust more than intended. client_of fails with UnknownPeer when the exchange has no peer, as with serve_connection on a custom transport.

Deploying behind a reverse proxy

std.http.server serves plain HTTP/1.1. For a public service, run it behind a reverse proxy that terminates TLS:

• Listen on loopback or a private address, never directly on the public interface. Then every peer is a proxy.
• Configure the proxy to overwrite, not append to, the header the program reads, or to append only its own peer address. Trust exactly the proxy’s addresses.
• Let the proxy enforce request rates and connection limits per client; the server’s limits are per connection and per worker.
• Keep config.limits and the body limits below the proxy’s, so the server answers oversized requests itself.
• Cookies default to Secure, which browsers send only over HTTPS. The program sees plain HTTP from the proxy; check scheme rather than assuming.

nginx and Caddy configurations tested in front of the server are in tests/fixtures/proxy/; the results are under reverse proxies.

Serving HTTPS directly

Without a proxy, serve TLS from the program with std.http.tls. Then it faces clients itself:

• Listen on the public address only through std.http.tls.serve; a plain serve on the same host should only redirect or be absent.
• No proxy is trusted, so leave Proxies empty: client_of returns the peer.
• Keep the server’s limits, timeouts and idle_connections tight; nothing in front enforces per-client rates.
• Cookies keep their Secure default, and the scheme is always https.

type HandlerError = std.http.HttpError | std.http.files.FilesError | std.http.cookie.CookieError | std.bytes.BytesError
type SetupError = std.http.server.ServerError | std.bytes.BytesError | AllocError | IoError | std.tls.TlsError | std.net.NetError

pub enum Route:
    Files
    Login
    Me

pub class Site:
    pub public: Root
    pub keys: list[Bytes]

    pub def handle(self, route: &Route, exchange: &mut Exchange[TlsTransport]) -> Result[(), HandlerError]:
        match route:
            case Route.Files:
                path = exchange.params[0]
                std.http.files.serve(exchange, &self.public, &path)?
            case Route.Login:
                mut head = std.http.response(204, "No Content")?
                std.http.cookie.add(&mut head, &std.http.cookie.signed_cookie(&self.keys, "session", "user42")?)?
                exchange.respond(head, &std.bytes.empty()?)?
            case Route.Me:
                found = std.http.cookie.cookies(&exchange.request.headers, 8)?
                match std.http.cookie.find_signed(&found, &self.keys, "session")?:
                    case Some(user):
                        exchange.send_text(200, &user)?
                    case Nothing:
                        exchange.send_text(401, "sign in first")?
        Ok(())

def main() -> Result[(), SetupError]:
    mut table = std.http.server.router[Route]()?
    table.add("GET", "/s/{path...}", Route.Files)?
    table.add("POST", "/login", Route.Login)?
    table.add("GET", "/me", Route.Me)?
    keys = [std.bytes.from_text(&read_text("/etc/site/session.key")?)?]?
    site = Site(std.fs.root(&std.path.from_text("/srv/site")?)?, keys)
    tls = std.tls.server_config(&read_text("/etc/site/chain.pem")?, &read_text("/etc/site/key.pem")?)?
    listener = std.net.listen(&std.net.parse_address("0.0.0.0:443")?, 128)?
    stop = CancellationToken()?
    report = std.http.tls.serve(listener, tls, &table, &site, &std.http.server.config(), &stop)?
    Ok(())

tests/test_std_http_files.rs runs this program on loopback with the test certificates in tests/fixtures/tls.

Static files

std.http.files serves files below a Root. It opens the request path one segment at a time through directory handles, refuses .., hidden names, links and anything but regular files, and answers ETag and If-Modified-Since; see its limits (no Range, no directory index). A reverse proxy can still serve static files when it is already in front.